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and  practitioners  identify  appropriate  analytic  approaches  for 
any  given  decision  situation.  An  attempt  is  made  to  suggest  a 
taxonomic  framework  for  codifying  the  state-of-the-art  of 
decision  analysis,  a language  for  expressing  *matching 
generalizations*  which  associate  the  appropriate  analytic 
option  to  a particular  situation.  This  language  has  three 
main  components,  each  of  which  has  an  exhaustive  numerical 
coding  scheme.  . 

The  first  component  is  a "situation  taxonomy,"  listing 
about  one  hundred  dimensions  of  a situation  that  might  be 
relevant  to  a particular  analytic  choice.  These  dimensions 
include:  the  stakes  involved  in  a decision;  the  reaction 

time  available;  and  the  clarity  with  which  options,  probable 
consequences,  and  values  are  perceived. 

The  second  component  is  an  "analysis  taxonomy,"  according 
to  which  about  one  hundred  decision-analytic  choices  can  be 
located  in  an  "analytic  option  space."  Dimensions  of  the 
analytic  taxonomy  include:  how  much  decision  analysis  is 

undertaken,  how  it  is  used,  what  type  of  model  structure  is 
involved,  and  what  technique  for  probability  assessment  or 
consequence  evaluation  is  employed. 

The  third  component  is  a "performance  measure  taxonomy," 
listing  about  thirty  measures  of  effectiveness  which  can 
characterize  the  analytic  options.  The  same  taxonomy  can 
also  be  used  to  describe  a situation  by  expressing  the  relative 
importance  of  the  performance  measures  in  the  situation. 
Performance  measure  dimensions  include:  enhanced  logical 

reasoning,  cost,  speed,  convenience,  and  facilitated  communi- 
cation. This  component  serves  as  a mediating  factor, 
implicit  or  explicit,  in  matching  analysis  to  situation. 

In  this  research  effort,  we  have  attempted  to  identify  a 
few  important  and  plausible  matching  generalizations  based  on 
the  experience  of  practicing  decision  analysts.  A few  analytic 
options  were  selected  to  represent  thousands  of  possibilities 
and  to  facilitate  generalizations  about  when  they  should  be 
exercised  in  the  form  of  a taxonomy  matching.  A U.S.  decision 
on  whether  to  export  high-technology  items  to  the  Soviet  Bloc 
is  analyzed  by  using  the  taxonomic  matching  framework.  Other 
illustrative  material  is  also  used  throughout  the  report. 
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SUMMARY 


This  report  attempts  to  develop  a conceptual  framework 
within  which  the  state-of-the-art  of  applied  decision  analysis 
can  be  codified.  The  framework  consists  of  a three-way 
taxonomy:  decision  situations,  analytic  options,  and  per- 

formance measures.  Within  this  framework,  a tentative  and 
illustrative  set  of  practical  guidelines  is  presented  to 
help  decision  analysis  users  and  practioners  identify  appro- 
priate analytic  approaches  for  any  given  decision  situation. 

Decision  makers  and  decision  analysis  practitioners  are 
both  faced  with  problems  of  determining  what  type  of  analysis, 
if  any,  to  apply  to  a given  decision  situation.  The  standard 
approach  in  such  cases  has  been  for  the  decision  maker  or 
analyst  to  evaluate,  case  by  case,  any  special  features  of 
the  situation  and  for  him  to  apply  informal  experience  and 
judgment,  reflecting  his  perception  of  the  state-of-the-art 
of  decision  analysis,  to  determine  the  appropriate  aDDroach 
for  the  situation. 

In  this  report,  an  attempt  is  made  to  suggest  a taxonomic 
framework  for  codifying  the  state-of-the-art  of  decision 
analysis,  a language  for  expressing  "matching  generalizations" 
which  associate  the  appropriate  analytic  option  to  a particular 
situation.  This  language  has  three  main  components,  each  of 
which  has  an  exhaustive  numerical  coding  scheme. 

The  first  component  is  a "situation  taxonomy,"  listing 
about  one  hundred  dimension's  of  a situation  that  might  be 
relevant  to  a particular  analytic  choice.  These  dimensions 
include:  the  stakes  involved  in  a decision;  the  reaction 

time  available;  and  the  clarity  with  which  options,  probable 
consequences,  and  values  are  perceived.  These  dimensions 
collectively  define  a "situation  space"  in  which  a situation 
can  be  unambiguously  located. 

The  second  component  is  an  "analysis  taxonomy,"  according 
to  which  about  one  hundred  decision-analytic  choices  can  be 
located  in  an  "analytic  option  space."  Dimensions  of  the 
analytic  taxonomy  include:  how  much  decision  analysis  is 

undertaken,  how  it  is  used,  what  type  of  model  structure  is 
involved,  and  what  technique  for  probability  assessment  or 
consequence  evaluation  is  employed. 

The  third  component  is  a "performance  measure  taxonomy," 
listing  about  thirty  measures  of  effectiveness  which  can 
characterize  the  analytic  options.  The  same  taxonomy  can 


also  be  used  to  describe  a situation  by  expressing  the  rela- 
» tive  importance  of  the  performance  measures  in  the  situation. 

Performance  measure  dimensions  include:  enhanced  logical 

reasoning,  cost,  speed,  convenience,  and  facilitated  communi- 
cation. This  component  serves  as  a mediating  factor,  implicit 
or  explicit,  in  matching  analysis  to  situation. 

In  this  research  effort,  we  have  attempted  to  identify  a 
few  important  and  plausible  matching  generalizations  based  on 
the  experience  of  practicing  decision  analysts.  The  general- 
izations do  not  attempt  to  be  exhaustive,  much  less  definitive. 
To  do  so  would  require  formally  codifying  the  entire  current 
state-of-the-art  of  decision  analysis.  Instead,  an  attempt 
has  been  made  to  select  a few  analytic  options  to  represent 
thousands  of  possibilities  and  to  generalize  about  when  they 
should  be  exercised  in  the  form  of  a taxonomy  matching.  (They 
reflect  the  kind  of  technical  judgment  that  an  experienced 
decision  analyst  might  communicate  to  a new  decision  analyst) . 

Two  examples  (representing  informally  the  coded  form  of 
this  report)  are: 

Generally  consider  using  decision  analysis  on  a 
single,  non-recurring  problem  only  if  the  "stakes"  are 
more  than  a hundred  thousand  dollars,  if  the  options 
are  clearly  visualized,  and  if  the  decision  maker  has 
more  than  usual  difficulty  in  deciding  among  them. 

Use  the  Delphi  method  for  eliciting  group  probabil- 
ities only  if  there  is  a high  degree  of  disparity  in  the 
status  of  group  members  and  if  the  sources  of  information 
available  to  them  are  rather  similar. 

A U.S.  decision  on  whether  to  export  high-technology 
items  to  the  Soviet  Bloc  is  analyzed  by  using  the  taxonomic 
matching  framework.  Other  illustrative  material  is  also  used 
throughout  the  report. 

In  addition  to  direct  use  of  this  "universal"  situation 
taxonomy,  the  notion  of  "specific"  situation  taxonomies, 
which  refer  to  specific  fields  such  as  naval  tactics,  is 
introduced.  Each  specific  taxonomy  can  be  more  or  less 
loosely  associated  with  the  universal  situation  taxonomy  and 
hence  matched  to  analytic  options.  The  use  of  a specific 
taxonomy  is  illustrated  in  a characterization  of  government- 
versus-business  decision  situations. 
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At  the  current  state  of  development,  the  situation 
f taxonomy  is  too  complex  to  be  applied  routinely  in  unfolding 

decision  situations.  The  idea  of  using  a procedure  for  pro- 
gressively applying  the  taxonomy  so  that  it  will  operate  like 
a succession  of  progressively  finer  screens  is  presented. 

Further  developments  suggested  include:  developing 

► "filtering"  decision  rules  (possibly  for  implementation  on 

an  interactive  computer)  that  permit  sequential  (and  econom- 
ical) elicitation  of  situation  characteristics,  refining  the 
taxonomy  matching  for  selected  analytic  choices,  and  developing 
parochial  taxonomies  and  matchings  for  selected  fields  (e.g., 
contingency  planning  for  naval  tactics) . 
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PREFACE 


The  present  paper  is  a technical  report  on  a study 
conducted  over  a two-year  period,  1974-1976.  The  study 
represents  a continuing  task  within  the  decision  analysis 
research  program  sponsored  by  Dr.  Martin  A.  Tolcott,  Engineer- 
ing Psychology  Programs,  Office  of  Naval  Research. 

The  Decisions  and  Designs,  Incorporated  (DDI)  project 
team  consisted  of  Rex  V.  Brown,  the  principal  investigator, 
and  Jacob  W.  Ulvila,  under  the  general  direction  of  Cameron  R. 
Peterson,  DDI  Technical  Director.  The  report  was  edited  by 
Michael  L.  Hays,  DDI  Publications  Manager. 

Since  this  project  is  visualized  as  a tentative  first 
step  in  a major  undertaking,  at  its  grandest,  a codification 
of  the  art  of  decision  analysis,  we  would  very  much  appreciate 
any  comments  or  suggestions  which  may  help  produce  something 
like  a definitive  framework  for  ourselves  and  others  to  work 
with.  If  responses  are  encouraging  enough,  this  material 
may  be  adapted  at  some  stage  for  general  publication. 
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1 . 0 INTRODUCTION 


Decision  makers  and  analysts  who  are  in  a position  to 
recommend  the  use  of  decision  analysis  are  faced  with  problems 
of  determining  what  type  of  analysis,  if  any,  to  apply  to  a 
given  decision  situation.  Currently,  the  decision  maker  or 
analyst  must  rely  upon  his  own  experience  and  must  make  an 
implicit,  case-by-case  judgment  of  the  appropriate  analytic 
approach  for  each  decision  situation.  This  report  is  an 
initial  attempt  to  improve  this  process  by  providing  a 
codification  of  the  combined  judgments  of  experienced  decision 
analysts . 


1 . 1 Object  and  Background 

Decision  analysis  might  be  defined  as  "a  technology  of 
making  up  your  mind."  Central  to  this  technology  is  the 
argument  that,  given  a set  of  reasonable  behavior  assumptions, 
any  person's  optimal  decision  can  be  unambiguously  determined 
from  quantified  measures  of  his  judgments,  attitudes,  and 
perceptions,  by  maximizing  his  subjective  expected  utility. 
This  technology  has  enjoyed  substantial  vogue  in, the  litera- 
ture of  management  science  over  the  past  decade.  As  with 
most  new  technologies,  however,  the  rate  at  which  decision 
analysis  has  been  applied  in  practice  has  been  much  slower 
than  the  rate  at  which  its  logical  underpinnings  have  been 
developed.  A reason  for  this  lag  is  the  lack  of  guidance 
that  exists  for  identifying  the  most  appropriate  decision- 
analytic  option  to  be  used  in  a given  situation. 


The  operational  military  commander  or  other  decision 
maker  has  available  to  him,  in  principle,  a moderately  well 
developed  array  of  decision-making  aids  based  on  decision 
analytic  theory. ^ However,  it  is  by  no  means  always  readily 
apparent  to  him  or,  indeed,  to  his  technical  advisors  when 
and  where  he  can  most  advantageously  use  these  tools  and 
how.  In  theory,  these  tools  can  be  applied  to  an  almost 
indefinitely  variable  degree,  to  any  situation  involving 
choice  or  inference. 


The  broad  purpose  of  this  project,  then,  is  to  guide 
the  decision  maker  and,  to  a lesser  degree,  his  technical 


1See , for  instance,  Raiffa  (1968),  Schalifer  (1969),  and 
Brown,  Kahr,  and  Peterson  (1974) ; and  Handbook  for  Decision 
Analysis  (1973) . 
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See,  for  example,  Section  4 of  Brown  et  al . 


(1974)  . 


staff,  in  determining  the  appropriate  use  of  decision- 
analytic  tools.  This  paper  is  an  attempt  to  produce  some 
illustrative  practical  guidelines  on  the  use  of  decision 
analysis.  More  importantly,  it  is  an  attempt  to  facilitate 
the  advancement  of  the  stcte-of -the-art  of  decision  analysis 
by  proposing  a framework  in  which  more  definitive  guidelines 
can  be  developed.  It  takes  into  account  earlier,  less 
extensive,  attempts  to  classify  or  relate  situations  and 
analytic  approaches. 3 

1.2  Terms  of  Reference 


The  main  objective  of  this  paper  is  to  encourage  and 
facilitate  the  codification  of  the  current  state-of-the-art 
of  applied  decision  analysis.  In  order  to  meet  this  objective, 
this  paper  attempts  to  suggest  a convenient  taxonomy  to 
classify  a particular  decision  situation  and  a taxonomy  of 
analytical  options  for  addressing  those  situations,  together 
with  such  guidance  as  can  be  given  for  an  appropriate  matching 
of  the  two. 

Because  an  attempt  to  classify  completely  every  conceiv- 
able decision-analytic  option  and  every  possible  decision 
situation  is  an  unmanageable  task  (with  thousands  of  taxonomic 
characteristics  for  both  analytic  options  and  situations) , 
the  focus  of  this  study  is  necessarily  restricted.  For  now, 
we  have  chosen  to  restrict  our  main  attention  to  those  rela- 
tively few  major  dimensions  which  appear  to  be  sufficient  to 
characterize  the  important  class  of  matching  generalizations 
that  are  useful  from  the  standpoint  of  a decision  maker; 
only  secondary  consideration  is  given  to  developing  guidelines 
for  a decision  analyst. 

This  restriction  affects  both  the  analytic  taxonomy  and 
the  situation  taxonomy.  The  scope  of  the  analytic  taxonomy 
is  reduced  because  serious  attention  is  devoted  to  the  dozens 
of  analytic  options  that  are  of  interest  to  the  decision 
maker  rather  than  to  the  thousands  of  analytic  options  that 
are  of  interest  to  the  analyst.  The  content  of  situation 
taxonomy  is  affected  because  the  most  appropriate  characteri- 
zation of  a decision  situation  for  specifying  the  correct 
options  for  the  decision  maker  may  differ  from  that  appropriate 
for  specifying  the  correct  options  for  the  analyst.  For 
example,  the  fact  that  the  decision  situation  calls  for 


3For  example,  by  Howard  (1968) , Howard  et  a^.  (1975  and  1976) , 

MacCrimmon  and  Taylor  (1973),  Payne  et  al.  (1974),  and 
Von  Winterfeldt  (1975) . 
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formal  documentation  of  the  reasons  behind  a choice  may  be 
very  relevant  to  deciding  whether  or  not  to  use  decision 
analysis  at  all  (a  decision  maker's  option)  but  almost  irrele- 
vant to  whether  simulation  or  backwards  induction  be  used  in 
the  conduct  of  the  decision  analysis  (an  analyst's  option). 

Primary  attention  is  also  restricted  to  providing  univer- 
sal situation  characterizations  that  allow  matching  rules  to 
be  developed  for  decision  situations  in  general  rather  than 
for  a particular  class  of  decision  situations,  such  as  those 
faced  by  a businessman  or  those  faced  by  a Navy  task  force 
commander.  This  restriction  affects  the  content  of  the 
situation  taxonomy.  In  particular,  the  situation  taxonomy 
contains  situation  characterizations  that  facilitate  matching 
analytic  options  and  situations,  rather  than  ones  that  make 
it  easy  for  a decision  maker  to  classify  a situation.  Ulti- 
mately, to  gain  the  maximum  usefulness  from  the  universal 
taxonomy,  it  will  be  necessary  to  develop  taxonomies  that 
enable  situations  to  be  more  easily  classified.  This  task 
is  very  large  because  decision  makers  in  different  fields, 
such  as  business  or  Navy  tactics,  tend  to  classify  decision 
situations  in  different  ways,  with  the  result  that  a specialized 
taxonomy  may  be  needed  for  each  field.  For  example,  a business- 
man may  classify  decision  situations  as  investment  decision 
situations,  marketing  decision  situations,  and  so  forth.  A 
Navy  task  force  commander,  on  the  other  hand,  may  classify 
decisions  as  crisis  decisions  or  wartime  decisions.  Thus, 
the  specific  taxonomy  that  will  serve  to  relate  the  decision 
maker's  preferred  method  of  classifying  situations  to  the 
method  used  in  the  universal  taxonomy  will  be  different  for 
the  businessman  and  for  the  task  force  commander.  The  develop- 
ment of  the  specific  taxonomies,  however,  is  beyond  the  scope 
of  the  present  paper. 

The  present  paper  develops  a set  of  matching  generaliza- 
tions in  a universal  taxonomy  and  illustrates  these  general- 
izations in  the  context  of  an  actual  decision  situation,  a 
determination  of  what  level  of  embargo  on  the  role  of  computers 
to  the  Soviet  Bloc  should  be  favored  by  the  U.S.  government. 

In  addition,  the  concept  of  a specific  taxonomy  is  notionally 
illustrated  in  the  context  of  a Navy  task  force  commander's 
decision. 

1. 3 Research  Approach 

In  constructing  a taxonomy  of  decision-making  situations, 
it  is  natural  to  inquire  whether  principles  exist  for  the 
construction  of  taxonomies  in  general.  The  most  widespread 
use  of  taxonomic  procedures  has  been  in  the  biological 
sciences,  and  much  has  been  written  on  appropriate  ways  to 
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produce  classification  systems.  However,  most  of  the 
stress  in  these  writings  is  on  the  construction  of  automatic 
classification  procedures.  With  these  procedures,  a dissim- 
ilarity measure  is  defined  for  pairs  of  objects  that  are  to 
be  classified,  and  then  a clustering  method  is  used  to 
create  classes.  We  do  not  feel  that  it  is  feasible  to 
create  such  automatic  classification  procedures  for  decision- 
making situations,  at  least  not  at  this  early  stage  in  our 
approach  to  the  problem,  since  the  similarity  or  dissimilar- 
ity between  two  decision-making  situations  is  a matter  of 
direct  judgment.  This  similarity,  at  least  at  present,  is 
unlikely  to  be  derivable  from  a numerical  formula.  Other 
areas  of  study  have  used  classification  schemes,  but  there 
do  not  appear  to  be  any  principles  in  the  literature  for  the 
construction  of  taxonomies  that  we  can  apply  directly  to 
decision-making  situations. 

The  logic  underlying  the  taxonomic  matching  of  analytic 
options  to  situations  is  based  upon  the  idea  of  maximizing 
the  value  of  the  decision  analysis  performed  with  due  allowance 
for  its  cost.  The  formal  principles  of  this  idea, 5 are  as 
yet  cumbersome  to  apply  in  practice.  Accordingly,  the 
approach  taken  in  this  paper  is  rather  informal,  though 
implicitly  it  should  be  consistent  with  those  principles. 

The  essential  research  approach  used  here  was  to  examine 
the  state-of-the-art  as  perceived  by  the  authors  and  others 
and  seek  to  codify  the  implicit  taxonomic  matching  it  represents. 
The  illustrative  methodological  generalizations  used  were 
derived  from  the  wide  range  of  practical  experience  of 
analysts  at  Decision  and  Designs,  Inc.  In  particular,  the 
generalizations  represent  the  judgments  of  two  analysts  (Rex  V. 
Brown  and  Cameron  R.  Peterson) , both  of  whom  have  actively 
applied  decision  analysis  to  a wide  range  of  government  and 
industry  problems  over  the  last  eight  years. 

In  addition,  the  situation  taxonomy  was  checked  and 
amplified  by  examining  a large  number  of  practical  Naval  and 
other  decision  situations.  Experienced  judgment  was  used  to 
identify  and  prioritize  their  important  distinguishing 
characteristics.  Similarly,  experienced  judgment  was  applied 
to  the  classification  of  analytic  options  with  a view  toward 
making  matching  generalizations. 


4 

See,  for  example,  Sokal  and  Sneath  (1963)  or  Jardine  and 
Sibson  (1971) . 

5Watson  and  Brown  (1975/1) . 
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2.0  A TAXONOMIC  FRAMEWORK 


In  this  study,  an  attempt  is  made  to  suggest  a taxonomic 
framework  that  provides  the  terms  for  codifying  the  state- 
of-the-art  of  decision  analysis.  These  terms  provide  a 
language  for  expressing  "matching  generalizations,"  which 
identify  the  appropriate  analytic  alternatives  in  any  parti- 
cular situation.  This  language  has  three  main  components. 

The  first  component  is  a "situation  taxonomy,"  which 
classifies  a situation  in  terms  relevant  for  choosing  analytic 
alternatives.  Representative  dimensions  of  the  situation 
taxonomy  include:  the  stakes  involved  in  a decision,  the 

reaction  time  available,  and  the  clarity  with  which  options, 
probable  consequences,  and  values  are  perceived.  The  complete 
set  of  these  dimensions  define  a "situation  space"  in  which 
a situation  can  be  unambiguously  located.  The  set  of  situa- 
tion dimensions  is  summarized  in  Appendix  A. 

The  second  component  is  an  "analysis  taxonomy,"  which 
locates  decision-analytic  choices  in  an  "analytic-option 
space."  Representative  dimensions  of  the  analytic  taxonomy 
include:  how  much  decision  analysis  is  undertaken;  what 

role  is  assigned  to  it;  what  model  structure  is  used;  and 
what  kind  of  probability  assessment  or  consequence  evaluation 
is  applied.  A summary  of  the  dimensions  of  the  analytic 
taxonomy  is  presented  in  Appendix  B. 

The  third  component  is  a "performance  measure  taxonomy," 
(Appendix  C)  which  allows  matching  situations  and  analytic 
alternatives.  The  state-of-the-art  of  applied  decision 
analysis  can  be  thought  of  in  terms  of  mapping  one  space 
onto  another,  that  is,  matching  a situation  with  an  analytic 
alternative  (see  Appendix  F) . The  validation  of  such  a 
mapping  requires  either  implicit  or  explicit  mediation  of 
the  performance  measure  taxonomy  since  its  dimensions  can  be 
associated  with  situations  as  priorities  and  with  analysis 
alternatives  as  properties.  That  is,  although  analytic 
options  can  always  be  matched  to  situations,  the  matchings 
cannot  be  justified  or  validated  except  by  using  the  perfor- 
mance measure  taxonomy.  Representative  dimensions  of  the 
performance  measure  taxonomy  include:  enhanced  logical 

reasoning,  lowered  cost,  and  facilitated  communication. 

2 . 1 Situation  Taxonomy 

The  situation  taxonomy  contains  dimensions  for  classifying 
decision  situations  in  a manner  that  facilitates  identifying 
the  appropriate  amount  of  decision  analysis  and  the  appropri- 
ate decision-analytic  choices  for  a given  situation.  This 
taxonomy  includes  only  those  situation  distinctions,  a 
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subset  of  all  possible  distinctions,  impacting  on  those 
performance  measures  (see  Section  2.3)  that  are  important 
for  determining  the  appropriate  amount  or  type  of  decision 
analysis. 

Table  2-1  contains  a summary  of  questions  (dimensions) 
in  the  situation  taxonomy.  Appendix  A presents  a detailed 
description  of  the  taxonomic  questions  and  possible  responses 
(values)  which  between  them  define  the  situation  characteristics. 

Each  situation  characteristic  is  numerically  coded  for 
easy  reference;  thus,  S112  refers  to  the  question,  "Expected 
Number  of  Occurrences,"  and  S1123  refers  to  the  answer, 

"Expected  Number  of  Occurrences  = 2"  (defined  in  Appendix  A) . 

The  "S"  is  used  to  distinguish  the  situation  characteristics 
from  other  taxonomies.  The  final  digit  of  the  code  is 
always  a specific  response  to  the  question  indicated  by  the 
other  digits. 

The  items  of  Table  2-1  enclosed  in  parentheses  are 
questions  that  distinguish  only  the  type  of  decision  .analysis , 
not  the  appropriate  amount  of  decision  analysis,  to  be 
performed.  Thus,  these  should  be  ignored  in  a first  pass  at 
characterizing  a situation. 

2. 2 Analytic  Options 

The  analytic  taxonomy,  summarized  in  Table  2-2  and 
detailed  in  Appendix  B,  attempts  to  identify  many,  but  not 
all,  decision-analytic  choices  available  to  decision  makers 
and  analysts  and  to  classify  them  in  a manner  that  facilitates 
a matching  of  specific  alternatives  to  a decision  situation. 
Future  references  to  the  analytic  options  will  be  preceded 
by  the  letter  "A."  Analogously  to  the  situation  taxonomy, 
the  first  digits  signify  the  choice  to  be  made;  the  last 
digit  in  the  code  corresponds  to  a specific  alternative. 

For  instance,  "comprehensive/partial  analysis?"  will  be 
referenced  as  A22  and  "comprehensive  analysis"  as  A221. 

As  with  the  situation  taxonomy,  any  particular  analysis 
will,  in  general,  be  characterized  by  a number  of  choices. 

2.3  Performance  Measures 


The  following  list  of  performance  measures  attempts  to 
capture  the  dimensions  sufficient  to  determine  the  amount 
and  type  of  decision  analysis  to  use.  To  the  extent  that 
the  relative  importance  of  these  measures  can  be  determined 
directly  in  a given  situation,  the  situation  taxonomy  presented 
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Decision  Substance 


Decision  Process 


11  BASIC  SITUATION 

21  REACTION  TIME 

111 

current/contingent  choice 

211 

minutes 

112 

expected  # of  occurrences 

212 

hours 

(113 

operating/information  act) 

213 

days 

119 

other  basic  situation 

214 

months 

12  OPTIONS 


22  ANALYTIC  PROCESSES 


(121 

broad/narrow) 

221 

# of  input  sources 

122 

clear/fuzzy 

222 

analytic  team 

123 

complexity  of  decision 

22'3 

constraints  on  analytic 

options 

method 

124 

radical/adaptive 

224 

documentation 

125 

static/dynamic 

229 

other 

129 

other  options 

13  DECIDABILITY 


23  ORGANIZATIONAL  PROCESSES 


14 


15 


16 


(231 

initiation) 

131 

difficulty  of  choice 

232 

responsibility 

132 

unfamiliarity 

(233 

coordination) 

133 

key  considerations 

234 

justification 

134 

other  decidability 

235 

con trover siality 

236 

performance  control 

237 

rational-actor  model 

238 

risk  attitude 

STAKES 

239 

other  organizational 

141  resources  committed 

142  cost  swing 

143  value  swing 

144  maximum  option  impact 

145  expected  irrationality  cost 
149  other  stakes 


OUTCOME  VALUATION 


processes 


24  DECISION  MAKER  CHARACTERISTIC 

241  role  in  organization 

242  personal  characteristics 
249  other 


151 

difficulty  of  net  valuation 

25  RESOURCES  AVAILABLE 

(152 

# of  value  dimensions) 

(153 

measurable  value?) 

251 

computational  facilities 

(154 

natural  combinability  of 

252 

staff 

values) 

253 

decision  analysis  expert 

(155 

timing) 

254 

availability  of  decision 

(156 

difficulty  of  component 

maker 

valuation) 

(255 

availability  of  assessors 

159 

other  outcome  uncertainty 

256 

dollars  available 

259 

other  resources 

OUTCOME  UNCERTAINTY 

161  # of  uncertainties 

162  determinability 

163  high/low  uncertainty 

164  subsequent  acts 
(165  type  of  evidence) 

166  hindsight  monitoring 
169  other  outcome  uncertainty 


19  OTHER  DECISION  SUBSTANCE 


29  OTHER  DECISION  PROCESSES 
291  negotiation 


Table  2-1:  SITUATION  TAXONOMY  - SUMMARY 


I 


1 USER'S  OPTIONS 


il  USE  DECISION  ANALYSIS 
AT  ALL? 


111  intuitive 

112  decision  analysis 
119  other 


12  DOLLAR  AMOUNT  OF  ANALYSIS 


INPUT  STRUCTURE 


31  UNCERTAINTY 

311  explicit  modeling 

312  time  horizon 

313  subsequent  acts 

31 4 event  sequency 

ordering 

315  detail  level 

316  degree  of  grouping 


ELICITATION  TECHNIQUE 

451  for  discrete 

probabilities 

452  for  continuous 

variables 

453  for  values 

454  use  group 

elicitation? 


121  low 

122  medium 

123  high 


13  ROLE  OF  DECISION  ANALYSIS 

131  private/public  aid 

132  prescribed/optional 

133  contingent/current 

analysis 

134  optimization/display 

135  communication 


14  ORGANIZATION 

141  analysis  source 

142  input  source 

143  "vest-pocket"  relation 
to  decision  maker? 


15  RESOURCES 


321  comprehensive? 

322  decomposed? 

323  partial  list? 

324  single  index? 

325  function? 


33  SPECIAL  FORMS 

331  Markov 

332  Pareto 

333  linear  programming 


INPUT  SPECIFICATION 


41  DECISION  OPTIONS 

411  specificity  of  definition 
419  other 


51  SPECIFICATION 

511  preferred  decision 

512  single  value  for 

each  option 

513  value  distributions 
519  other 


52  DISPLAY  FORMAT 

521  graphic 

522  computer 
529  other 


ANALYTIC  DEVICES 

531  use  simulation 
539  other  analytic 
devices 


6 MODEL  MANAGEMENT 


151  use  a decision 

analysis  expert? 

152  use  a computer? 


2 MODEL  APPROACH  OPTIONS 


21  COMPLEXITY  OF  ANALYSIS 


211  simple 

212  moderately  complex 

213  very  complex 


22  COMPREHENSIVE/PARTrAL  ANALYSIS 


221  comprehensive 

222  partial 


23  APPROXIMATE  ANALYSIS? 


421  scenarios 

422  specific 
429  other 


43  VALUE  CRITERIA 


230  no 

231  yes 


431  units 

432  base 

433  evaluation  date(s) 

for  time  stream 


44  INDIRECT  ASSESSMENTS 


441  conditioned  assess- 

ment model? 

442  Bayesian  updating? 

443  decomposed  assess- 

ment model? 

444  regression? 

449  other 


61  MODEL  DYNAMICS 

611  combining 

612  pooling 

613  sequential  modeling 

614  decision  option 

scanning 

615  input  iteration 


62  CONTINGENT  ANALYSIS  INPUT 
SEQUENCE 

621  values 

622  priors 

623  likelihood 

624  data 


Table  2-2:  ANALYTIC  TAXONOMY  - SUMMARY 
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above  is  unnecessary.  However,  because  it  is  typically  very 
difficult  to  determine  this  rating,  performance  measures 
must  serve  an  intermediate  purpose  in  matching  analytic 
options  to  situations. 

Table  2-3  contains  a summary  taxonomy  of  performance 
measures,  and  Appendix  C presents  a detailed  description  of 
each  measure.  In  future  references,  the  performance  measure 
numbers  will  be  preceded  by  the  letter  "P."  For  instance, 
"conceptual  completeness”  will  be  referenced  as  Pill.  By 
contrast  to  the  situation  and  analysis  taxonomies,  specific 
values  on  the  dimensions  have  not  been  coded  as  the  final 
digit. 


t- 
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11  LOGIC  OF  CHOICE  31  ACTIVITY  PRECEDING  CHOICE 

PROCESSES 


111 

112 

conceptual  completeness 
effective  disagqregation 

311 

good  environment 

113 

sound  predictions 

monitoring 

114 

good  overall  logic 

312 

good  decision 

115 

scope 

identification 

119 

other 

313 

good  option  generation 

314 

good  pre-analysis  of 
anticipated  decisions 

QUALITY  OF  INPUT 

319 

other 

121 

good  data  gathering 

32  ACTIVITY  FOLLOWING  CHOICE 

122 

good  management  of  staff/ 

PROCESSES 

expertise 

123 

posing  meaningful  questions 

good  decision  communi 

124 

good  overall  input  quality 

321 

129 

other 

cation 

322 

good  hindsight  eval- 

uation 

2 Time 

and  Cost 

323 

effective  implementa- 

tion 

329 

other 

21  ELAPSED  TIME 


33  ORGANIZATIONAL  AND  OTHER 

211 

short  elapsed  modeling 

NON- 

"CHOICE  SPECIFIC" 

212 

fast  input  assignment 

IMPACTS 

213 

fast  calculation 

214 

fast  interpretation 

215 

short  overall  net  elapsed 

331 

improved  information 

time 

332 

improved  command , 

219 

other 

control,  and 

communication 

333 

improved  body  of 

COSTS 

applied  precepts 

339 

other 

221 

inexpensive 

analysis 

222 

inexpensive 

input  assignment 

223 

inexpensive 

calculation 

224 

inexpensive 

overall 

229 

other 

Table  2-3: 

PERFORMANCE  MEASURES- 

SUMMARY 
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3.0  MATCHING  HIGHLIGHTS 


In  Section  3.1,  we  shall  generate  matching  hypotheses 
about  the  circumstances  that  justify  the  use  of  any  decision 
analysis  based  on  our  applied  experience.  In  doing  so,  we 
shall  uncover  and  identify  the  relevant  situation  taxonomy 
that  these  generalizations  imply.  Throughout  this  effort, 
the  emphasis  is  on  ensuring  that  the  generalizations  address 
the  proper  situation  taxonomic  considerations  rather  than  on 
ensuring  that  the  specific  generalizations  are  correct.  It 
is  certainly  not  argued  that  these  generalizations  are  ready 
for  adoption  as  they  stand  as  tenets  of  recommended  decision- 
analytic  practice. 

There  is,  in  principle,  a different  situation  taxonomy 
for  every  level  of  decision-analytic  alternatives  (despite  a 
substantial  amount  of  overlap).  For  example,  to  decide 
whether  to  use  decision  analysis  at  all  requires  character- 
izing a decision  situation  according  to  stakes,  reaction 
time,  and  the  like,  whereas  deciding  whether  to  use  the 
Delphi  method  for  group  elicitation  requires  such  information 
as  whether  the  informant  group  has  heterogeneous  status  and 
homogeneous  information. 

It  is  not  feasible  to  itemize  exhaustively  the' union  of 
all  such  special-purpose  sets  of  characteristics,  and  it  is 
even  less  feasible  to  generate  a complete  set  of  matching 
generalizations  for  all  decision-analytic  alternatives  and 
their  associated  situation  taxonomies.  To  attempt  any  kind 
of  completeness  is  to  attempt  to  codify  the  whole  state-of- 
the-art  of  decision  analysis. 

Accordingly,  since  our  aim  is  more  modest,  we  shall 
explore  only  a limited  set  of  matching  generalizations  that 
appear  either  important  in  their  own  right  or  are  in  some 
sense  representative  of  a complete  taxonomy.  The  primary 
motivation  is  to  generate  a useful,  somewhat  general-purpose 
situation  taxonomy  that  suggests  interesting,  even  provocative 
selective  generalizations  about  the  state-of-the-art.  This 
preliminary  and  partial  treatment  may  provide  a starting 
point  for  developing  definitive  situation  taxonomies  and 
matching  generalizations.  In  Section  3.2,  we  extend  the 
matching  concept  to  the  determination  of  the  form  of  decision 
analysis  to  use. 

3 . 1 Amount  of  Decision  Analysis 


In  this  section,  we  restrict  attention  to  the  most 
common  and  perhaps  most  critical  analytic  choice:  how  much 


decision  analysis,  if  any,  to  perform.  This  choice  will  be 
analyzed  in  two  steps.  First,  decision  analysis  will  be 
characterized  in  terms  of  the  performance  measures  introduced 
in  Section  2.3.  Second,  the  analytic  choice  of  how  much 
decision  analysis  to  perform  will  be  related  through  perfor- 
mance measures  to  the  situation  taxonomy  of  Section  2.1. 

Throughout  this  section,  references  to  the  situation 
taxonomy  will  be  preceded  by  the  letter  "S"  and  references 
to  the  performance  measure  taxonomy  will  be  preceded  by  the 
letter  "P."  These  items  are  explained  in  detail  in  Appendices 
A and  C respectively. 

3.1.1  Performance  characteristics  of  decision  analysis  - 
The  validation  of  any  of  the  matching  generalizations  in 
Sections  3.1.2,  3.2.1,  3.2.2,  and  3.2.3  will  be  through  the 
explicit  or  implicit  mediating  role  of  performance  measures 
summarized  in  Section  2.3  above. 

This  section  explains  how,  in  general,  decision 
analysis  performs  along  the  different  performance  measures. 

In  all  cases,  the  comparisons  are  between  decision  analysis 
and  conventional  decision-making  practice  as  exemplified  by 
intuitive  or  informal  decision-making  procedures.  Decision 
analysis  is  not  being  compared  with  other  quantitative 
approaches  such  as  operations  research  or  management  science 
techniques  although  we  recognize  that  these  techniques  are 
often  appropriate.  In  all  cases,  the  statements  presented 
are  broad  generalizations  and  there  are  exceptions.  The 
generalizations  represent  highly  personal  judgments  of  the 
authors,  but  they  are  grounded  in  substantial  experience. 

The  following  performance  measure  generalizations 
are  presented  in  the  order  in  which  the  measures  appear  in 
Table  2-3,  "Performance  Measure  Summary,"  rather  than  in 
order  of  importance. 

Decision  analysis  generally  performs  well  in  the 
first  performance  area,  quality  of  decision  (PI) .1  That 
is,  to  the  extent  that  this  performance  measure  is  important, 
a large  amount  of  decision  analysis  is  indicated.  Under  the 
logic  of  choice  category  (Pll) , decision  analysis  performs 
well  on  most  sub-categories.  In  particular,  use  of  more 
decision  analysis  generally  leads  to:  more  effective  dis- 

aggregation of  the  problem  into  manageable  sub-problems 
(P112) , sounder  predictions  based  upon  the  available  data 


^"Parenthetical  references  are  to  the  performance  measure 
taxonomy  detailed  in  Appendix  C and  summarized  in  Table  3-2. 


(P113) , and  better  overall  logic  (P114).  However,  it  is  not 
clear  whether  more  decision  analysis  leads  to  a more  conceptually 
complete  consideration  (Pill) . On  one  hand,  using  decision 
analysis  tends  to  increase  understanding,  but,  on  the  other 
hand,  using  decision  analysis  can  lead  to  serious  omissions. 

Thus,  on  balance,  the  use  of  more  decision  analysis  leads 
to  better  overall  logic  (P114) . 

In  the  quality  of  input  category  (P12) , decision 
analysis  performs  well  on  all  sub-categories.  In  particular, 
the  use  of  more  decision  analysis  will,  frequently,  promote: 

o Better  data  gathering  (P121) ; 

o Better  management  of  staff  and  expertise  (P122) ; 
and 

o More  meaningful  questions  (P123) ; 

all  of  which  result  in  better  overall  input  quality  (P124) . 

While  the  quality  of  decision  (Pi)  usually 
increases  with  an  increased  use  of  decision  analysis,  the 
time  and  costs  involved  (P2)  also  increase. 

First,  using  decision  analysis  will  generally 
increase  elapsed  time  (P21)  on  a first  pass,  which  includes 
slower  first-pass  modeling  time  (P2111) , slower  first-pass 
input  assignment  (P2121) , and  slower  first-pass  calculation 
(P2131) . It  is  not  clear  whether  or  not  the  first-pass 
interpretation  speed  (P2141)  will  be  affected  by  the  amount 
of  decision  analysis.  Thus,  overall  first-pass  elapsed  time 
(P2151)  will  be  longer. 

Additional  passes  of  the  analysis  will,  however, 
generally  be  faster  when  decision  analysis  is  used.  While 
it  is  not  clear  that  this  is  the  case  with  the  elapsed 
modeling  time  (P2112) , it  is  the  case  with  input  assignment 
(P2122) , calculation  (P2132) , and  interpretation  (P2142) . 

Thus,  overall  time  for  additional  passes  (P2152)  is  reduced. 

The  net  for  the  total  overall  elapsed  time  (P2153) , including 
both  first  and  additional  passes,  is  generally  greater  when 
decision  analysis  is  used. 

In  the  cost  category  (P22) , more  decision  analysis  always 
raises  the  cost  of  the  decision  analysis.  Greater  analytic 
cost  occurs  in  all  of  the  sub-categories:  analysis  cost  ( P 2 2 1 ) , 

input  assignment  cost  (P222) , calculation  cost  (P223) , and 
overall  cost  (224)  . It  occurs  for  both  the  first  pass  and 
additional  passes.  It  should  be  emphasized  that  cost  as  used 
in  this  context  refers  to  the  cost  of  the  decision  analytic 
effort,  not  to  gains  or  costs  that  might  accrue  to  decision 
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outcomes  from  having  applied  more  or  less  decision  analysis 
to  the  problem.  The  trade-off  of  analytic  costs  (how  much 
analysis)  with  the  possible  impact  of  that  analysis  is  one  of 
the  uses  of  the  proposed  taxonomy. 

In  the  third  area,  other  considerations  (P3) , 
the  effects  of  decision  analysis  cannot  be  generalized  for 
most  sub-categories.  Using  more  decision  analysis  does  not 
have  a clearly  generalizable  effect  on  environment  monitoring 
(P311) , decision  identification  (P312) , option  generation 
(P313) , pre-analysis  of  anticipated  decisions  (P314) , imple- 
mentation (P323) , information  processing  (P331) , or  command, 
control,  and  communication  (P332) . However,  use  of  decision 
analysis  will  generally  improve  decision  communication 
(P321) , hindsight  evaluation  (P322) , and  the  body  of  applied 
precepts  (P333) . 

The  amount  of  decision  analysis  indicated  in  a 
situation  can  be  determined  by  a direct  assessment  of  the 
relative  importance  of  the  performance  measure.  That  is,  to 
the  extent  that  the  performance  measures  favoring  decision 
analysis  are  more  important  than  those  not  favoring  decision 
analysis,  more  decision  analysis  is  indicated.  However, 
since  it  is  typically  difficult  to  determine  the  relative 
importance  of  the  performance  measures  directly,  it  is 
usually  more  convenient  to  use  the  following  situation 
taxonomy  to  determine  the  proper  amount  of  decision  analysis. 

3.1.2  Situations  favoring  the  use  of  decision  analysis- 
The  above  section  explained  that  a situation  characterization 
in  terms  of  performance  measures  is  sufficient  to  determine 
the  appropriate  amount  of  decision  analysis.  However, 
because  this  characterization  is  often  difficult  to  obtain, 
the  situation  taxonomy  presented  in  Section  2.1  was  developed. 
This  taxonomy  is  a more  natural  way  to  classify  situations, 
but  it  is  not  directly  relatable  to  the  appropriate  amount 
of  decision  analysis.  2 

This  section  summarizes  the  most  important  aspects 
of  the  means  by  which  situations  can  be  related  to  the  appro- 
priate amount  of  decision  analysis  through  the  performance 
measures.  Appendix  D contains  a more  detailed  matching  of  how 
much  decision  analysis  to  use  in  a situation  which  addresses 
all  situation  dimensions  shown  in  Table  2-1.3  Unless  otherwise 


2 

The  matching  task  is  conceptually  analogous  to  the  formulation 
of  a regression  problem.  This  analogy  is  presented  in 
Appendix  F. 

^Situation  dimensions  are  detailed  in  Appendix  A. 


noted,  situations  that  support  the  use  of  decision  analysis 
also  support  the  use  of  more  decision  analysis.  (Whenever  a 
statement  is  made  that  a certain  situation  characteristic 
supports  the  use  of  decision  analysis,  it  implies  that  the 
more  that  an  actual  situation  exhibits  the  situational  charac- 
teristic, the  greater  the  quantity  of  decision  analysis  that 
is  supported.)  All  statements  are  guidelines  and  need  to  be 
viewed  as  a whole  rather  than  in  isolation.  Combined  effects 
of  several  dimensions  may  cause  decision  analysis  to  be  favored 
where  no  single  categorization  is  sufficient  to  make  that 
indication . 

As  shown  in  Table  3-1,  a large  amount  of  decision 
analysis  is  most  strongly  recommended  in  a situation  with: 

o Clear  options  (S1221)4 

o A difficult  choice  (S1312) 

o A choice  as  the  key  consideration  (S1334) 
o A maximum  option  impact  greater  than 
$10  million  (S1444) 

o Several  substantive  and  preference  sources 

(S2215) 

o Required  pre-  and  post-decision  justifications 

(S2343) 

o A decision  maker  who  is  familiar  with  decision 
analysis  (S24213)  . 

In  addition,  a large  amount  of  decision  analysis  is  supported, 
but  less  strongly,  by  a situation  with: 

o A current  choice  (Sill) 

o A recurring  choice  (S1124) 

o A difficult  valuation  (S1512) 

o Indeterminable  uncertainties  (S1621) 

o Months  of  reaction  time  (S214) 

o Organizational  processes  resembling  a rational- 

actor  model  (S2372 ) 

o Availability  of  many  computational  facilities 

(S2512) 

o A strong  staff  (S2522) 

o A high  degree  of  decision  maker  availability 
(S2542 ) 

o A high  degree  of  dollar  availability  (S2563) . 

An  example  illustrating  the  use  of  these  rules  is  presented 
in  Section  4.1.1  below. 


4 


Situation  dimensions  are  detailed  in  Appendix  A. 
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“ * S2512  Many  Computational  Facilities 

X 

£ •§  Available 

g 3 S2522  Strong  Staff  Available 

2 > S2542  Much  Decision  Maker  Availability 

06  < S2563  Greater  than  $50,000  Available 

X 

o £ * “ S24213  D.  M.  Has  Much  Familiarity  with 
® * * 2 Decision  Analysis 


Both  Pre  and  Post  Justification 
Rational  Actor  Model  is  Good 
Approx. 


S2215  Several  Input  Sources  for  Both 
Preferences  and  Substantive 
Inputs 


S214  Months  of  Reaction  Time 


S1621  Indeterminable  Uncertainties 


S1312  Very  Difficult  Net  Valuation 


51441  Maximum  Option  Impact  S100K  x 

51442  Maximum  Option  Impact  $100K-$5M  x 

51443  Maximum  Option  Impact  $5M-$10M  x 

51444  Maximum  Option  Impact  > $10M 


S 1 312  Difficult  Choice 
S1334  Key  Consideration  Choice 


S1221  Clear  Options 


Sllll  Current  Choice 
SI  124  More  than  two  expected  if 
of  Occurrences 
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The  following  paragraphs  summarize  the  reasoning 
behind  the  primary  situation  characterizations  that  support 
the  use  of  decision  analysis. 

Decision  analysis  can  improve  conceptual  completeness 
(Plix,  most  when  the  choice  options  are  clearly  specified 
(S1221 ) . 

Decision  analysis  can  provide  the  greatest 
improvement  in  the  decision  quality  (PI)  for  difficult 
choices  (S1312) . When  choice  is  difficult,  informal  decision 
processes  leave  much  room  for  improvement,  which  can  be 
filled  by  using  decision  analysis. 

Decision  analysis  can  generally  improve  choice 
logic  (Pll) . Thus,  decision  analysis  is  indicated  in  situa- 
tions in  which  choice  is  the  key  consideration  (S1334). 

The  stakes  involved  in  the  decision  (S14)  are 
the  single  most  important  situation  classification  for 
determining  the  appropriate  amount  of  decision  analysis.  In 
general,  the  cost  of  the  analysis  (P22)  becomes  less  important 
as  the  stakes  increase.  Thus,  decision  analysis  is  indicated 
in  high  stakes  situations.  As  a rough  rule  of  thumb,  we 
suggest  that  the  maximum  option  impact,  the  dollar  equivalent 
difference  in  outcome  attributable  to  the  choice  of  the  best 
decision  over  the  worst  reasonable  decision,  should  be  on 
the  order  of  $100,000  (S1442)  to  justify  any  decision  analysis. 

Of  course,  decision  analysis  is  more  strongly  indicated  if 
the  maximum  option  impact  is  greater  than  this  threshold. 

Conventional,  intuitive  decision  practice  has 
difficulty  organizing  input  from  a variety  of  sources. 

Decision  analysis,  however,  provides  a method  for  organizing 
a diverse  set  of  inputs.  In  particular,  decision  analysis 
can  improve  the  quality  of  both  input  (P12)  and  choice  logic 
(Pll)  when  multiple  input  sources  (S2215)  exist.  Input 
quality  is  especially  improved  through  the  effective  manage- 
ment of  staff  and  expertise  (P122) , and  choice  logic  is 
especially  improved  through  effective  disaggregation  of  the 
decision  problem  (P112) . 

Decision  analysis  can  provide  a vehicle  for  good 
decision  communication  (P321) , which  is  required  in  situations 
in  which  decision  justification  is  needed  (S2343)  . 

In  a situation  in  which  the  decision  maker  is 
familiar  with  decision  analysis  (S24213) , the  analysis  can 
provide  its  greatest  benefits,  especially  the  effective 
implementation  of  a decision  (P323) , the  effective  management 
of  staff  expertise  (P122) , and  good  overall  decision  logic 
(P114)  . 
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Secondary  considerations  that  support  the  use  of 
decision  analysis  are  as  follows: 

Current  choice  (Sllll)  favors  the  use  of  decision 
analysis  because  it  is  certain  that  the  analysis  could  be 
used.  Since  decision  analysis  is  a tool  to  aid  in  decision 
making,  its  value  as  an  aid  is  reduced  in  contingent  situations 
(S1112)  in  which  it  is  uncertain  that  the  decision  will  ever 
occur. 

A decision  that  is  expected  to  recur  (S1124) 
supports  the  use  of  decision  analysis  because  the  cost  (P22) 
per  analysis  is  reduced.  The  amount  of  advantage  gained  by 
recurrence,  of  course,  depends  upon  both  the  expected  frequency 
of  recurrence  and  the  similarity  of  the  recurring  situations 
because  the  similarity  determines  the  amount  of  modification 
needed  before  the  analysis  can  be  used  again.  The  expected 
number  of  occurrences  is,  in  general,  closely  related  to  the 
current/contingent  question  (Sill) , explained  above,  and  the 
two  dimensions  should  be  considered  simultaneously.  For 
instance,  a current  decision  that  will  occur  only  once 
should  be  considered  the  same  as  a contingent  decision  that 
has  a 50%  chance  of  occurring  twice.  Both  of  these  situations 
have  an  expected  occurrence  rate  of  one. 

Decision  analysis  can  improve  choice  logic  (Pll) 
in  situations  in  which  valuation  is  difficult  (S1512). 

Decision  analysis  can  improve  choice  logic  (Pll) 
in  situations  in  which  uncertainties  are  indeterminable 
(S1621) , in  the  sense  that  it  is  difficult  to  determine  what 
the  appropriate  probability  distribution  should  be  (regardless 
of  whether  it  has  a large  or  small  dispersion) . 

More  time  is  required  to  perform  a decision 
analysis  than  to  use  informal,  intuitive  decision-making 
techniques.  Thus,  situations  with  long  reaction  time  (S214) 
support  the  use  of  decision  analysis. 

Since  decision  analysis  assumes  a rational-actor 
model,  a situation  that  is  a good  approximation  of  the  model 
(S2372)  supports  the  use  of  decision  analysis. 

Good  computational  facilities  tend,  in  general, 
to  reduce  the  cost  of  analysis  (P221).  Thus,  decision 
analysis  is  supported  where  good  computation  facilities 
exist  (S2512) . 

A strong  staff  (S2522)  is  most  likely  to  perform 
a decision  analysis  correctly  and  thus  improve  the  overall 
logic  of  the  choice  (P114) . 
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If  a decision  maker  is  unavailable  during  the 
analysis  time,  the  improvement  in  choice  logic  (Pll)  and 
input  quality  (P12)  provided  by  decision  analysis  is  reduced. 
Thus,  decision  analysis  is  supported  when  the  decision  maker 
is  available  (S2542). 

When  a large  resource  is  available  for  an  analysis 
(S2563) , the  cost  of  the  analysis  becomes  unimportant.  Thus, 
the  use  of  decision  analysis  is  supported  by  a large  available 
resource. 


This  section  has  highlighted  the  key  situation 
dimensions  that  support  the  use  of  decision  analysis.  A more 
thorough  examination  of  the  relationship  between  the  situation 
taxonomy  and  the  amount  of  indicated  decision  analysis  is 
presented  in  Appendix  D. 

3. 2 Examples  of  Type  of  Analysis 

In  this  section,  the  matching  concept  of  Section  3.1  is 
extended  to  the  determination  of  the  form  of  decision  analysis 
to  use  in  a given  situation.  This  matching  is  summarized  in 
Table  3-2.  This  section  will  highlight  the  summary  by 
presenting  three  examples: 

o Situations  favoring  the  use  of  a contingent  choice 
analysis , 

o Situations  favoring  the  use  of  a computer  to 
perform  the  analysis,  and 

o Situations  favoring  the  use  of  the  Delphi  assess- 
ment technique. 

Appendix  E contains  a detailed  description  of  all  of  the 
matching  generalizations  shown  in  Table  3-2.  A case  illustration 
of  this  part  of  the  taxonomy  matching  in  use  appears  in 
Section  4.1.2. 

3.2.1  Contingent  analysis  (A1331)5  (analysis  in 
advance  of  a decision  that  is  expected  to  arise  in  the 
future)  - Performance  characteristics:  a contingent  choiceg 

analysis  requires  a large  amount  of  time  initially  (P2151). 


^Analytic  options  are  described  in  Appendix  B. 
^Performance  measures  are  described  in  Appendix  C. 
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However,  it  provides  a fast  response  on  subsequent  uses 
(P2152) . In  addition,  in  a situation  characterized  by  a 
short  reaction  time  (S211)^,  performing  a contingency  analysis 
in  advance  can  improve  choice  logic  (Pll) , input  quality 
(P12) , and  control  (P332). 

Situations  favoring  contingent  analysis  - The 
most  important  situation  considerations  for  determining 
whether  to  perform  a contingent  analysis  include  the  number 
of  occurrences  (S112),  the  stakes  (S144),  and  the  reaction 
time  (S21) . 

In  general,  contingent  choice  analysis  is  indicated 
in  situations  in  which  its  cost  (P22)  is  justified  by  either 
a large  number  of  occurrences  (S1124)  or  by  large  stakes 
(S1444).  As  a rule  of  thumb,  a contingent  analysis  should 
not  be  performed  unless  the  expected  number  of  occurrences 
is  at  least  one  (P1122)  (for  example,  occurring  once  with 
certainty  or  occurring  twice  with  a 50%  probability) . It  is 
also  necessary  to  consider  the  similarity  as  well  as  the 
number  of  the  occurrences.  The  more  similar  the  situations 
are,  the  more  the  contingent  analysis  is  justified.  (As  a 
rule  of  thumb,  three  similar  recurrences  are  roughly  equivalent 
to  a single  identical  recurrence.)  For  a good  pre-analysis 
(P314) , it  is  critically  important  that  the  actual  decision 
is  predictable,  that  is,  involves  the  same  considerations 
that  are  modelled. 

As  a guideline,  the  expected  option  impact  over 
a number  of  occurrences  should  exceed  $10  million  (S1444)  to 
justify  the  cost  of  a contingent  analysis.  Brown  et^  al . 

(1975)  present  an  illustration  of  this  guideline  in  a Navy 
task  force  commander's  decision  situation. 

In  a situation  with  a short  reaction  time  (S211) , 
a pre-analysis  of  contingencies  can  improve  decision  quality 
(PI)  by  providing  a logical  framework  that  considers  all 
available  data. 

In  addition,  a contingent  analysis  is  supported, 
but  less  strongly,  by  a situation  with  clear  options  (S1221) 
and  several  value  sources  (S2213) . Clear  options  (S1221) 
allow  a contingent  analysis  to  provide  a good  pre-analysis 
of  the  anticipated  decision  (P314) . Clear  options  also 
enhance  predictability  (see  above) . Contingent  choice 
analyses  can  improve  command,  control,  and  communication 
(P332)  in  situations  that  require  several  value  sources  to 
be  considered  (S2213). 


^Situation  dimensions  are  described  in  Appendix  A. 
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3.2.2  Use  of  computer  (A1521)  - Performance  character- 
istics : A computerized  analysis  is  generally  time-consuming 

and  costly  to  set  up  (P2131  and  P2231)  but  is  fast  and 
inexpensive  to  run  (P2132  and  P2232) . However,  when  an 
analysis  is  complex,  a computer  may  not  increase  the  initial 
time  or  cost. 

Situations  favoring  the  use  of  a computer  include 
a recurring  situation  JS1124 ) or  a situation  with  large 
stakes  (S1444) . Since  a computer  offers  faster  and  less 
expensive  performance  on  subsequent  passes,  its  use  is 
supported  by  a large  expected  number  of  occurrences  (S1124) . 

Large  stakes  argue  for  the  use  of  a computer  for  two  reasons. 
First,  large  stakes  justify  the  large  costs  needed  for  a 
computerized  analysis.  Second,  a situation  with  large 
stakes  generally  involves  a complex  analysis.  In  this  case, 
a computerized  analysis  may  not  be  more  expensive  than  a 
manual  analysis. 

The  following  situations  support  the  use  of  a 
computer,  but  less  strongly  than  those  mentioned  above. 

A computer  analysis  requires  at  least  several 
days  of  anticipatory  reaction  time  (S213) . However,  since  a 
computerized  analysis  runs  quickly,  only  minutes  of  execution 
reaction  time  (S211)  are  needed. 

Some  staff  support  (S2521)  is  needed  to  program 
a computerized  analysis. 

The  costs  (P22)  and  time  (P21)  of  a computerized 
analysis  are  not  significantly  worse  than  the  alternative  of 
a non-computerized  analysis  if  the  situation  has  clear, 
complex  options  (S1221  and  S1236);  many  determinable  uncertainties 
(S1613  and  S1623) ; and  many  measurable  values  (S1523  and 
S1532 ) . 

A computer  is  also  recommended  in  conjunction 
with  a complex  analysis  (A213)  and  with  simulation  (A5311) . 

3.2.3  The  Delphi  Technique  (A45411)  (a  group  elicitation 
technique  for  probabilities)  - Performance  characteristics; 

The  Delphi  technique  involves  poolinq  opinions  of  several 
probability  assessors  while  allowing  only  limited  interaction 
among  them. 

Situations  favoring  the  use  of  the  Delphi  Technique 
include  only  those  with  several  uncertainty  sources  (S2214) . 
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The  Delphi  technique  limits  interaction  among 
assessors  and  provides  the  assessors  with  anonymity.  Thus, 
the  Delphi  technique  removes  inhibition  and  allows  better 
use  of  staff  expertise  (P122)  in  cases  in  which  the  status 
of  the  assessors  is  heterogeneous.  However,  if  data  is 
heterogeneous,  then  the  Delphi  technique  reduces  input 
quality  (P12)  by  inhibiting  useful  interactions.  (See 
Fischer  [1975]  for  a further  evaluation  of  the  Delphi  technique.) 


4.0  MODE  OF  USE 


The  primary  objective  of  the  taxonomy  of  decision 
situations  presented  above  is  to  provide  the  decision 
analysis  user  with  a convenient  device  for  recognizing 
promising  situations  in  which  to  use  decision  analysis,  and 
to  guide  him  in  making  the  kinds  of  analytic  choices  illustrated 
in  Sections  3.1  and  3.2.  At  the  current  state  of  development, 
the  situation  taxonomy,  though  incomplete,  is  too  complex  to 
be  applied  routinely  in  unfolding  decision  situations. 

What  is  ultimately  needed  is  a procedure  for  progressively 
applying  the  taxonomy  so  that  it  will  operate  like  a succession 
of  progressively  finer  screens.  The  answers  to  a few  routinely 
applied  questions,  perhaps  related  to  the  size  of  the  stakes 
involved,  the  time  available  to  make  a decision,  the  clarity 
of  the  choices,  and  the  complexity  of  the  outcomes,  will 
signal  which  situations  should  be  seriously  considered  for 
decision  analysis.  For  those,  a second  level  of  more  sharply 
focused  questions  will  be  used  to  make  a definitive  decision 
on  whether  to  use  decision  analysis  and,  if  so,  what  role 
the  analysis  will  serve.  After  resolving  these  issues,  even 
more  sharply  focused  questions  will  be  used  to  determine  the 
detailed  planning  of  the  analysis.  The  screening  would 
continue  until  the  exact  analytic  options  are  identified,  or 
until  informal  technical  judgment  is  sufficient  to  designate 
the  type  of  analysis. 

4 . 1  Application  of  Universal  Taxonomy 

As  explained  in  Section  1.2,  this  paper  has  developed  a 
universal  situation  taxonomy  to  aid  in  matching  decision- 
analytic  options  to  decision  situations.  Section  4.1.1 
below  illustrates  how  the  universal  taxonomy  should  be 
applied  to  a decision  situation  in  order  to  determine  the 
appropriate  amount  of  decision  analysis,  and  Section  4.1.2 
illustrates  the  use  of  the  universal  taxonomy  in  choosing 
the  type  of  analysis  for  the  same  situation. 

4.1.1  Amount  of  decision  analysis  to  use  on  export 
control  decision  - In  the  summer  of  1973,  a senior  staff 
member  of  the  President's  Council  of  International  Economic 
Policy  (CIEP ) had  to  make  a recommendation  to  the  President 
on  the  level  of  embargo  for  computers  sold  to  the  Soviet 
Bloc.  If  approved  by  the  President,  in  consultation  with  the 
Departments  of  State  and  Defense,  this  recommendation  would 
provide  the  basis  for  the  U.S.  position  on  COCOM  embargo 
policy.  COCOM,  an  international  coordinating  committee 
representing  the  major  western  powers  and  Japan,  is  charged 
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with  controlling  the  export  of  computers  and  other  strategic 
items  to  communist  countries.  The  essential  decision  by 
COCOM  was  to  specify  where,  in  terms  of  computing  power,  to 
set  the  "easy  access  line"  for  computers. 


Six  weeks  before  his  recommendation  was  due,  the 
CIEP  staffer  was  faced  with  a very  large  array  of  reports 
from  experts  on  the  complex  issues  involved  in  the  decision. 
These  issues  included  the  impact  of  the  decision  on  military 
threat,  U.S.  computer  sales,  the  attitude  of  COCOM  allies, 
and  other  economic  and  political  considerations.  Although 
the  staffer  had  some  sense  of  where  the  "easy  access  line" 
should  be  set,  he  felt  uneasy  about  his  ability  to  properly 
digest  and  interpret  the  complex  mass  of  data  available  to 
him  within  the  short  time  available. 


The  first  question,  then,  was  how  much,  if  any, 
decision  analysis  the  staffer  should  have  done,  and,  secondly, 
what  specific  form  the  decision  analysis  should  have  taken. 


Table  4-1  indicates  how  the  matching  highlights 
shown  on  Table  3-1  might  be  used,  informally,  to  conclude 
that  a substantial  amount  of  decision  analysis  should  be 
used.  It  can  be  seen  that  all  but  three  of  the  seventeen 
situation  characteristics  favoring  a large  amount  of  decision 
analysis  are  present  in  this  case,  including  all  but  one  of 
seven  key  characteristics. 


Specifically,  proceeding  from  left  to  right,  we 
see  that  the  choice  was  a current  one,  and  that  the  decision 
situation  is  expected  to  recur  (the  issue  is  expected  to 
come  up  for  re-examination  every  two  years  or  so) . The 
options  are  somewhat  clear,  in  the  sense  that  the  theoretically 
rich  and  complex  option  space  could  be  reduced  to  six  specific 
alternatives  without  gross  over-simplification.  The  choice 
was,  however,  not  particularly  difficult  in  that  the  decision 
maker  was  rather  sure  that  the  right  choice  lay  between 
three  of  the  six  options.  The  decision  maker  saw  his  key 
problem  as  choosing  from  among  perceived  options  (rather 
than  making  predictions  or  determining  what  the  options  are, 
for  instance).  All  measures  of  stakes  were  very  high,  the 
maximum  option  impact  was  on  the  order  of  hundreds  of  millions 
of  Federal  budget  dollars.  The  net  valuation  of  possible 
outcomes  was  very  difficult  (trade-offs  of  economic,  political 
and  military  considerations  were  required) , but  there  was  no 
unusually  great  indeterminability  of  uncertainties.  The 
reaction  time  was  on  the  order  of  a couple  of  months.  There 


I 


25 


were  several  different  sources  of  input  for  the  various 
types  of  uncertainty  and  prediction,  all  of  which  were 
different  from  the  sources  of  value  judgment.  The  decision 
required  justification  in  a formal  report  to  the  President 
and  in  verbal  presentations  to  representatives  of  industry 
and  government  agencies  both  before  and  after  the  decision 
was  to  be  made.  The  decision  processes  of  government  for 
this  decision  are  reasonably  "rational"  (in  that  reasoned 
argument  is  a large  part  of  the  persuasion  process) . The 
decision  maker  was  highly  familiar  with  decision  analysis 
(having  taught  the  subject  at  a major  univeristy) . Computational 
facilities  and  staff  skills  were  more  than  adequate.  The 
decision  maker  was  able  to  spend  large  amounts  of  time 
participating  in  the  analysis  on  a regular  basis.  However, 
the  budget  available  was  quite  modest,  on  the  order  of 
$30,000. 


At  this  stage  of  taxonomy  development,  no  formal 
decision  rule  is  proposed  for  determining  routinely  what 
level  of  decision  analysis  should  be  used.  In  this  case, 
however,  it  is  clear  that  any  plausible  decision  rule  would 
indicate  a large  amount  of  decision  analysis,  since  only  one 
of  the  seven  key  situation  characteristics  is  absent  (difficult 
choice) , and  only  two  of  the  other  ten  favorable  situation 
characteristics  are  absent  (indeterminable  uncertainty  and 
large  budget).1  However,  since  only  a small  budget  ($30,000) 
for  analysis  was  available,  its  size  alone  easily  determined 
the  dimensions  of  the  project  without  any  need  for  a formal 
decision  rule. 

4.1.2  Type  of  decision  analysis  to  use  on  export  control 
decision  - The  following  discussion  indicates  the  appropriate 
type  of  analysis  for  the  export  control  problem.  This 
discussion,  which  treats  each  analytic  option,  should  be 
considered  in  conjunction  with  Table  3-2  above. 

The  analysis  was  used  in  a display  rather  than 
in  an  optimization  mode  (A1342)  because  the  scope  of  the 
analysis,  at  the  client's  request,  was  not  comprehensive 
(A222) ; certain  political  dimensions  of  value  such  as 
impact  on  detente  were  deliberately  excluded  from  the  analysis. 
The  display  of  the  decision's  impact  on  those  dimensions  of 
value  that  were  explicitly  modeled  (economic,  military,  and 
so  forth) , however,  permitted  an  evaluation  of  how  large  the 
excluded  political  considerations  would  need  to  be  in  order 
to  change  the  preferred  option. 


1Watson  and  Brown  (1975/1)  presents  an  attempt  to  put  a value 
on  this  particular  analysis,  after-the-fact. 
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A computer  was  used  (A1521)  because  the  decision 
was  expected  to  recur  about  every  two  years  (S1124) , the 
stakes  were  high  (S1444) , and  many  input  iterations  (A61533) 
were  expected.  The  large  number  of  input  iterations  was 
indicated  because  this  was  a very  controversial  decision 
(S2352) . 


The  complexity  of  the  analysis  was  low  (A211) 
because  of  a need  to  communicate  (P321)  the  arguments  in 
order  to  justify  the  decision  (S2343)  and  because  few  of  the 
conditions  indicating  complexity  were  met  (see  Table  3-2) . 

In  particular,  only  one  probability  distribution  was  modeled 
(on  one  dimension  of  value--military  threat) . 

Subsequent  acts  were  not  explicitly  modeled 
(A3130)  because  they  did  not  have  enough  impact  (S1641)  to 
justify  any  explicit  modeling. 

On  the  other  hand,  value  was  decomposed  extensively 
(A3222)  into  its  four  major  dimensions  (U.S.  computer  sales, 
allied  goodwill,  military  threat,  and  other)  and  one  dimension, 
military  threat,  was  decomposed  into  sixteen  specific  variants. 
One  reason  was  that  the  sources  of  preference  judgment  were 
different  (S2213)  and  partly  because  the  decision  was 
controversial  (S2352) , especially  regarding  trade-offs  among 
value  dimensions  (for  example,  how  much  improvement  in 
computer  sales  will  compensate  for  a deteriorating  military 
threat. ) 


The  decompositions  were  all  linear  (A32511) 
because  of  a need  for  both  pre-  and  post- justifications 
(S2343) . 

The  value  scales  used  a "floating  base”  (A4322) 
because  the  technical  problems  of  valuation  of  each  dimension 
were  not  very  difficult  (S1510)  and  because  experienced 
staff  (S2522)  were  available  to  handle  the  greater  technical 
subtlety  involved. 

In  performing  the  calculations,  simulation  was 
not  needed  (A5310)  because  the  model  structure  was  not 
complex  (A211) . 

4.1.3  Sequential  taxonomy  matching  - Direct  use  of  the 
universal  taxonomy,  as  illustrated  above,  may  present  substantial 
problems  of  elicitation  if  all  potentially  relevant  characteristics 
are  involved.  Since  there  are  hundreds  of  situation  characteristics 
to  measure,  it  may  be  difficult  to  measure  any  one  of  them, 
let  alone  all  of  them.  That  is,  it  may  be  very  difficult  to 
locate  precisely  any  given  situation  in  situation  space. 
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However,  for  any  particular  choice  (for  example,  whether  to 
use  decision  analysis  at  all) , only  some  of  the  situation 
dimensions  are  relevant  and  even  all  of  those  may  not  need 
to  be  evaluated  to  make  a given  analytic  choice,  the  choice 
of  whether  to  use  decision  analysis  or  not  may  be  resolved 
on  the  grounds  of  stakes  alone  if  they  are  small  enough. 

For  example,  the  small  size  of  the  budget  (S2562)  and  the 
large  size  of  the  stakes  (S1444)  in  the  COCOM  Case  (see 
Section  4.1.1)  immediately  determined  the  amount  of  analysis. 

Thus,  a sequential  elicitation  of  situation 
characteristics  is  often  indicated.  In  principle,  a flow 
chart  could  be  constructed  to  control  the  sequence  of  situation 
characteristic  elicitation.  The  answers  to  each  situation 
question  either  indicate  an  analytic  choice  or  call  up  a new 
question.  Figure  4-1  shows  notionally  the  form  such  a flow 
chart  might  take,  but  it  does  not  represent  a considered 
proposal.  A complete  flow  chart  of  this  form  would  represent, 
in  effect,  a manual  of  applied  decision  analysis. 


Indirect  Taxonomy  Matching  Through  a Specific  Taxonomv 


Since  relevant  situational  characterizations  tend  to  be 
difficult  to  accomplish,  it  is  desirable  to  seek  mediating 
characterizations  that  enable  an  easy  characterization  to 
serve  as  a surrogate  for  a relevant  one.  For  this  reason, 
specific  taxonomies,  which  are  specific  to  a particular 
field  of  decision-making  and  which  are  based  upon  concepts 
that  are  familiar  in  the  field,  need  to  be  developed  in 
addition  to  the  universal  taxonomy  presented  above,  which  is 
not  specific  to  a particular  area  of  decision  making. 

Figure  4-2  illustrates  the  manner  in  which  a situation 
can  be  characterized  along  both  specific  and  universal 
taxonomies  in  order  to  determine  the  appropriate  decision- 
analytic  techniques.  The  decision  to  neutralize  an  enemy- 
held  island,  which  may  cause  a global  war,  might  be  charac- 
terized as  a "crisis"  situation  on  the  specific  taxonomy. 

This  characterization  is  associated  with  the  universal 
dimensions  of  "multiple  value  source,"  "controversial,"  and 
"high  stakes."  Through  matching  generalizations,  the  situation 
characterization  indicates  that  a substantial  scale  of 
decision  analysis,  requiring  explicit  disaggregation  of 
value,  is  indicated.  In  conjunction  with  other  specific  or 
universal  descriptions  of  the  situation,  this  matching 
indicates  specific  analytic  choices,  step-through  simulation 
model  structure,  and  decomposed  valuation. 

<> 
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Figure  4 2 

ILLUSTRATIVE  USE  OF  TAXONOMY  MATCHING 
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Table  4-2  illustrates  another  example  of  how  a specific 
taxonomy  matching  might  be  developed.  The  specific  character- 
istic of  business-versus-government  decisions  is  analyzed  to 
determine  how  much  decision  analysis  to  use.  Table  4-2 
indicates  that  government  decisions  are  more  frequently 
associated  with  the  universal  situation  characteristics 
favoring  decision  analysis  than  are  business  decisions. 

Table  4-2  suggests,  therefore,  that,  as  a general  rule 
(modified  of  course  by  particular  circumstances,  that  is, 
other  specific  or  universal  situation  characteristics) , 
government  decisions  favor  decision  analysis  more  than 
business  decisions  do. 

The  specific  taxonomy  examples  presented  above  are 
intended  only  to  present  the  idea  and  point  the  direction 
for  future  research. 2 


2 

In  addition,  a start  has  been  made  in  Sections  1,  2,  and 
6 of  Brown  et  al.  (1974)  on  developing  a specific 
taxonomy  for  naval  tactical  situations. 
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Universal  Taxonomy 
Characteristics  Favoring  DA 


SDecific  Characteristics 


Government  Business 


High  expected  number  of 
occurrences  (S1124) 


Difficult  choice  (S1312) 


High  stakes  (S14) 


Difficult  net  valuation  (S1512) 


Several  sources  for  both  substantive 
and  preference  inputs  (S2215) 


Controversial  (S2352) 


Need  for  post-decision  justification  X 

(S2343) 


Specific  matching  conclusion:  government  decisions  tend 

‘to  favor  the  use  of  decision  analysis  more  than  business 
decisions . 


Table  4-2:  APPLICATION  OF  SPECIFIC  TAXONOMY 
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5.0  CONCLUSIONS 

5 . 1 Implications  of  Work  Completed 

It  is  important  not  to  confuse  the  grand  design  of  the 
total  taxonomy  matching  task  with  the  modest  and  tentative 
beginning  which  this  report  represents.  What  is  proposed 
here  is  no  more  than  a conceptual  framework  with  some  notional 
substance.  Even  at  its  completion,  the  taxonomy  matching 
will  represent  essentially  a vehicle  for  communicating  the 
state-of-the-art  of  applying  decision  analysis  and  will  make 
only  incidental  contributions  to  the  state-of-the-art 
itself.  Possibly  the  structure  developed  will  encourage 
methodological  inquiry  simply  by  providing  a convenient 
vehicle  for  dissemination  of  the  results  of  such  inquiry. 

The  main  value  of  the  taxonomy,  however,  must  lie  in  being  a 
prompter,  a pointer,  and,  to  some  extent,  a stage  setter  for 
further,  more  substantive  research. 

However,  this  report  does  carry  some  of  the  tentative 
methodological  doctrine  that  a few  specific  decision  analysts 
at  Decisions  and  Designs,  Incorporated  have  developed.  That 
doctrine,  cast  in  this  format,  may  stimulate  some  controversy 
or  methodological  inquiry  especially  when  it  conflicts  with 
the  textbook  paradigms  of  decision  analysis  (for  example, 
the  modeling  of  subsequent  acts  as  events) . However,  the 
proper  vehicle  for  research  aimed  at  doctrinal  persuasion, 
at  least  in  the  first  instance,  is  in  topic  oriented  tech- 
nical reports,  not  here. 

5.2  Future  Research 


Since  this  project  is  only  a tentative  first  step 
toward  the  codification  of  decision  analysis,  we  are  well 
aware  that  much  remains  to  be  done  to  arrive  at  something 
even  approximating  a definitive  framework  for  ourselves  and 
others  in  the  field  to  work  with.  Toward  this  end,  we  are 
soliciting  comments  and  suggestions  from  colleagues  and 
interested  researchers.  If  responses  are  sufficiently 
encouraging,  it  is  planned  to  modify  this  material  and  ready 
it  for  general  publication. 

The  following  are  some  proposals  for  change  or  develop- 
ment under  current  consideration: 

1.  Re-order  items  in  the  performance  measure  taxonomy 
by  putting  the  "overall"  categories  first; 

2.  Include  "Don't  Know"  and  "Not  Modeled"  categories 
in  the  taxonomies; 

3.  Replace  the  last  (response)  digit  in  the  taxonomies 
with  a letter.  Thus,  "binary  options"  might  be 
referenced  as  "S123A"  instead  of  "S1231"; 
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4.  Expand  Section  A45,  elicitation  techniques; 

5.  Resolve  a possible  conflict  concerning  P112  and 

P123:  Is  the  performance  measure,  "effective  dis- 

aggregation," meaningful,  or  is  it  the  same  as 
"posing  meaningful  questions?"; 

6.  Review  terminology  to  be  used;  for  example,  should 
"substantive"  or  "uncertainty,"  describe  a type  of 
input  in  S221?;  and 

7.  Expand  the  bibliography. 

From  this  tentative  beginning,  a number  of  promising 
avenues  for  research  appear,  including  the  following: 

1 Refine  and  enrich  the  taxonomic  and  matching 
material  in  this  report  in  the  light  of  more 
thorough  investigation  of  the  same  issues,  and  of 
a wider  range  of  decision-analytic  experience 
(involving  organizations  other  than  DDI ) ; 

2.  Develop  specific  taxonomies  and  matching 
generalizations  for  particular  purposes,  for 
example,  a matching  taxonomy  to  select  decision 
aids  by  task  force  commanders  for  tactical  naval 
battle  decision  situations.  This  development 
could  be  a valuable  adjunct  to  an  ongoing  program 
sponsored  by  the  Office  of  Naval  Research  on 
Operational  Decision  Aidr, . This  particular  problem 
is  discussed  in  Appendix  A of  Payne  et  al.  (1974) 
and  Sections  1,  2,  and  6 of  Brown  et  al.  (1974); 

3.  Develop  an  analogous  analysis  to  that  presented  here 
directed  to  the  problem  of  matching  decision  situa- 
tions with  information  needs  (as  opposed  to  analytic 
approaches) . Such  an  inquiry,  or  a specific  version 
that  is  specific  to  Naval  tactics,  might  be  a valu- 
able adjunct  to  the  development  of  the  information 
component  of  a tactical  flag  command  center  (TFCC) 
of  the  type  currently  under  consideration  in  the 
U.S.  Navy; 

4.  Extend  generalizations  proposed  here  to  the  point 

of  proposing  specific  decision  rules  which  go  beyond 
the  simple  listing  of  favoring  characteristics; 

5.  Codify  the  state-of-the-art  decision  analysis  with 
respect  to  specific  technical  issues,  one  at  a time. 
This  codification  would  call  for  a richer  and  more 
definitive  expansion  of  sub-sections  of  Section  3.2, 
for  example,  expanding  an  individual  row  of  Table  3-2; 
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6. 


Develop  an  implementation  procedure  for  the  taxonomy 
matching,  for  example,  by  proposing  and  trying  out 
in  practice  specific  procedures  to  help  a decision 
maker  or  a technician  evaluate  candidate  decisions 
for  decision  analysis  (e.g.,  by  developing  a 
filtering  procedure  along  the  lines  of  Figure  4-1) ; 
and , 

7 Develop  interactive  computer  graphic  programs  to 

streamline  the  filtering  procedure  referenced  in  6 
above . 
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SITUATION  TAXONOMY 
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Decision  Substance 

2 

Decision  Process 

EASIC  SITUATION 

21 

REACTION  TIME 

in 

current/contingent  choice 

211 

minutes 

112 

expected  # of  occurrences 

212 

hours 

(113 

operating/information  act) 

213 

days 

119 

other  basic  situation 

214 

months 

OPTIONS 

22 

ANALYTIC  PROCESSES 

(121 

broad/narrow) 

221 

# of  input  sources 

122 

clear/fuzzy 

222 

analytic  team 

123 

complexity  of  decision 

223 

constraints  on  analytic 

options 

method 

124 

radical/adaptive 

224 

documentation 

125 

static/dynamic 

229 

other 

129 

other  options 

23 

ORGANIZATIONAL  PROCESSES 

DECIDABILITY 

(231 

initiation) 

131 

difficulty  of  choice 

232 

responsibility 

132 

unfamiliarity 

(233 

coordination) 

133 

key  considerations 

234 

justification 

134 

other  decidability 

235 

controversial ity 

236 

performance  control 

237 

rational-actor  model 

238 

risk  attitude 

STAKES 

239 

other  organizational 

processes 

141 

resources  committed 

142 

cost  swing 

143 

value  swing 

144 

maximum  option  impact 

24 

DECISION-MAKER  CHARACTERISTIC 

145 

expected  irrationality  cost 

149 

other  stakes 

241 

role  in  organization 

242 

personal  characteristics 

249 

other 

OUTCOME  VALUATION 

151 

difficulty  of  net  valuation 

25 

RESOURCES  AVAILABLE 

(152 

# of  value  dimensions) 

(153 

measurable  value?) 

251 

computational  facilities 

(154 

natural  combinability  of 

252 

staff 

values) 

253 

decision  analysis  expert 

(155 

timing) 

254 

availability  of  decision- 

(156 

difficulty  of  component 

maker 

valuation) 

(255 

availability  of  assessors 

159 

other  outcome  uncertainty 

256 

dollars  available 

259 

other  resources 

OUTCOME  UNCERTAINTY 

29 

OTHER  DECISION  PROCESSES 

161 

# of  uncertainties 

291  negotiation 

162 

determinabi lity 

163 

high/low  uncertainty 

164 

subsequent  acts 

(165 

type  of  evidence) 

166 

hindsight  monitoring 

169 

other  outcome  uncertainty 

19  OTHER  DECISION  SUBSTANCE 

Table  A 1 : SITUATION  TAXONOMY  - SUMMARV 


1  Decision  Substance 

The  following  dimensions  describe  the  substance  of  a 
decision  as  opposed  to  the  setting  in  which  the  decision  is 
to  be  made. 

11  Basic  Situation 

111  Current/contingent  choice  - Does  the  situa- 
tion demand  a choice  which  will  result  in  a 
current  or  contingent  action? 

1111  Current  - e.g.,  mobilize  NATO  forces 
immediately. 

1112  Contingent  - e.g.,  mobilize  NATO  forces 
if  and  when  the  probability  of  Warsaw 
Pact  forces  mobilizing  exceeds  .55. 

112  Expected  number  of  occurrences  - a prob- 
ability weighted  average  of  the  number  of 
times  that  the  decision  must  be  made  (in- 
cluding the  first  time) . 

1121  Expected  number  of  occurrences  1 

1122  Expected  number  of  occurrences  = 1 
(e.g.,  a current  choice  or  a contingent 
choice  that  will  occur  twice  if  it 
occurs  at  all  and  has  a 50%  chance  of 
occurring) . 

1123  Expected  number  of  occurrences  = 2 

1124  Expected  number  of  occurrences  2 

113  Operating/information  act  - Will  the  decision 
result  in  taking  an  operating  act  or  an  act 
to  seek  information? 

1131  Operating  - e.g.,  shoot  at  an  unidentified 
plane. 

1132  Information  - e.g.,  seek  information  on 
an  unidentified  plane's  identity. 

119  Other  basic  situations. 
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12  Options 
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121  Broad/Narrow  - Is  a commitment  required  at  a 
broad  or  narrow  level?  (A  narrow  level  is 
typically  a subset  of  a broad  one.) 

1211  Broad  - e.g.,  go  North  or  go  south. 

1212  Narrow  - e.g.,  go  to  Fairbanks,  Alaska, 
or  go  to  Miami,  Florida. 

122  Clear/Fuzzy  - Are  the  options  clearly  speci- 
fied or  not? 

1221  Clear  - e.g.,  selecting  one  of  five 
bidding  contractors. 

1222  Fuzzy  - e.g.,  selecting  criteria  to  use 
in  selecting  a contractor. 

123  Complexity  of  decision  options  - How  many 
decision  options  are  under  consideration? 

1231  Two  (a  binary  choice). 

1232  Three  to  twelve. 

1233  More  than  twelve  (discrete) . 

1234  One  scalar  - e.g.,  what  price  to  change. 

1235  Small  vector  - e.g.,  what  values  to 
assign  to  three  design  parameters. 

1236  Large  vector. 

1239  Other 

124  Radical/adaptive  - Is  the  decision  within  the 
realm  of  the  decision  maker's  current  practice? 

1241  Radical  - outside  of  the  realm  of  current 
practice  and  the  alternatives  are  very 
different,  (e.g.,  a decision  to  drop  a 
product) . 

1242  Adaptive  - involving  minor  changes  in 
current  practice  and  the  alternatives 
are  very  similar  (e.g.,  a decision  to 
raise  the  price  of  a product) . 
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-j  125  Static/dynamic  - Is  this  decision  final  or 
can  it  be  modified  over  time? 

1251  Static  - The  decision  is  to  be  made  now 
once  and  for  all  (e.g.,  a declaration  of 
war)  . 

1252  Dynamic  - The  commitment  can  be  modi- 
fied, possibly  continuously  (e.g., 
taking  an  increasingly  bellicose  stance 
toward  a potential  enemy,  maneuvering  a 
ship) . 

129  Other  options 
13  Ease  of  Decision 

131  Difficulty  of  choice  - Is  there  a lot  of  room 
for  improvement  over  informal  decision  making? 

1311  Routine  or  obvious  - Informal  techniques 
leave  little  room  for  improvement. 

1312  Difficult  - There  is  much  room  for  im- 
provement over  informal  decision-making 
techniques.  Decision  maker  is  perplexed 
about  what  to  do  and  there  is  a high 
"cost  of  confusion." 

132  Unfamiliarity  - How  foreign  is  the  choice  to 
the  decision  maker's  experience? 

1321  Low  - familiar  to  the  decision  maker's 
experience. 

1322  High  - foreign  to  the  decision  maker's 
experience . 

133  Key  consideration  - in  this  case,  what  is  the 
key  determinant  of  a good  decision? 

1331  Option  generation  - generating  a set  of 
viable  options. 

1332  Information  - gathering  information  to 
support  a choice. 

1333  Inference  - drawing  inferences  from 
available  information. 

1334  Choice  - choosing  from  among  a set  of 
options . 
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1339  Other  considerations. 

139  Other  decidability 
14  Stakes 

141  Resources  committed  - What  total  amount  of 
resources  are  involved  with  the  decision 
(e.g.,  value  of  facilities  engaged)? 

1411  Low  - less  than  $200,000. 

1412  Medium  - $200,000  to  $2  million. 

1413  High  - greater  than  $2  million. 

142  Cost  swing  - What  is  the  difference  in  cost 
between  the  least  expensive  and  most  expensive 
option? 

1421  Low  - less  than  $100,000. 

1422  Medium  - $100,000  to  $1  million. 

1423  High  - greater  than  $1  million. 

143  Value  swing  - What  is  the  difference  in  value 
between  the  best  and  worst  plausible  outcomes 
(regardless  of  option)? 

1431  - Less  than  $1  million. 

1432  - $1  million  to  $40  million. 

1433  - $40  million  to  $100  million. 

1434  - Greater  than  $100  million. 

144  Maximum  option  impact  - What  is  the  dollar 
equivalent  difference  in  outcome  attributable 
to  the  choice  of  best  decision  over  worst 
reasonable  decision? 

1441  - less  than  $100,000. 

1442  - $100,000  to  $5  million. 

1443  - $5  million  to  $10  million. 

1444  - Greater  than  $10  million. 


145  Expected  option  impact  - What  is  the  expected 
value  of  the  differences  attributable  to 
option  choice? 

1451  - less  than  $50,000. 

1452  - $50,000  to  $2  million. 

1453  - $2  million  to  $5  million. 

1454  - greater  than  $5  million. 

146  Expected  irrationality  cost  - What  is  the 
room  for  improvement  in  the  user's  decision 
making  process  (the  expected  cost  of  choosing 
an  option  other  than  that  indicated  by  an 
"impeccable"  decision  analysis)? 

1461  Low  - less  than  $150,000. 

1462  Medium  - $150,000  to  $500,000. 

1463  High  - greater  than  $500,000. 

149  Other  stakes 

15  Outcome  Valuation 

151  Difficulty  of  net  valuation  - How  difficult 
is  it  to  compare  the  attractiveness  of 
possible  outcomes?  (Specific  elements  of  this 
characterization  are  covered  below  in  items 
S152  - S155 . ) 

1510  Not  very  - e.g.,  for  a single  criterion 
with  a natural  metric. 

1511  Moderately. 

1512  Very  - e.g.,  for  multiple  conflicting 
intangible  criteria. 

152  Number  of  value  dimensions  - Is  a single 
value  criterion  (measure  of  effectiveness) 
sufficient  or  are  multiple  criteria  impor- 
tant? 

1521  Single 

1522  Few 

1523  Many 
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153  Measurable  value  - To  what  extent  does  a 
natural  metric  exist  for  the  value  dimen- 
sions? 

1530  No  natural  metric  - e.g.,  for  quality 
of  life. 

1531  Approximate  metric  - e.g.,  GNP  as  a 
metric  for  standard  of  living. 

1532  Exact  metric  - e.g.,  dollars  as  a metric 
of  income. 

154  Natural  combinability  of  value  - How  natural 
is  it  to  combine  the  dimensions  of  value  into 
a single  index? 

1541  Low  - e.g.,  in  the  case  of  many  con- 
flicting intangible  variables. 

1542  Moderate. 

1543  High  - e.g.,  for  different  additive 
components  of  a single  variable  such  as 
cost. 

1549  Other  - e.g.,  high  for  some  but  not 
for  all. 

155  Timing  - early/late  - Are  the  value  dimen- 
sions characterized  by  early  or  late  evalua- 
tion dates? 

1551  Early  - e.g.,  cost  of  a contract. 

1552  Late  - e.g.,  long-run  impact  on  U.  S. 
military  standing. 

156  Difficulty  of  component  valuation  - How 
difficult  is  it  to  value  outcomes  on  each 
criterion  scale? 

1561  Low 

1562  Medium 

1563  High 


159  Other  outcome  valuations. 
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Outcome  Uncertainty 


161  Number  of  uncertainties  - How  many  uncertain 
quantities  influence  the  decision? 

1611  One 

1612  Few 

1613  Many 

162  Determinability  - How  difficult  is  it  to 
determine  the  appropriate  probability  dis- 
tribution (s)  of  the  uncertain  quantities? 

1621  Low 

1622  Medium 

1623  High 

163  High/low  uncertainty  - What  is  the  "amount" 
of  uncertainty  in  the  uncertain  quantities 
(e.g.,  as  represented  in  the  "spread"  of  the 
probability  distribution)? 

1630  None 

1631  Low 

1632  Medium 

1633  High 

164  Subsequent  acts  - To  what  degree  do  subsequent 
acts  impact  the  decision  outcome? 

1641  Low 

1642  Medium 

1643  High 

165  Type  of  evidence  - What  is  the  nature  of  evi- 
dence available  for  resolution  of  uncertainty? 

1651  Historical  record. 

1652  Direct  judgment  only. 

1653  Indirect  judgment  - e.g.,  based  on 
conditioning  events. 


45 


166  Hindsight  monitoring  - How  accurately  and 
promptly  can  the  outcomes  be  determined  by 
hindsight? 

1661  Low  - e.g.,  having  to  wait  years  to  know 
whether  there  were  mines  at  the  entrance 
of  the  Dardanelles  in  1914. 

1662  Medium. 

1663  High  - e.g.,  discovering  within  hours 
that  a bombing  mission  was  successful. 

169  Other  outcome  uncertainties 


19  Other  Decision  Substance 
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Decision  Process 


In  terms  of  the  user’s  analytic  options  (as  opposed  to 
the  technician's  options),  the  organizational  and  personal 
setting  within  which  a decision  is  to  be  made  may  be  of  even 
greater  importance  than  the  specific  substance  of  the  decision. 
Thus,  we  may  be  interested  in  the  personal  characteristics 
of  the  decider,  the  organizational  setting  within  which  he 
operates,  or  external  constraints  on  the  decision  process. 

21  Reaction  Time  - How  much  reaction  time  is  available 

between  the  time  that  the  decision  is  recognized 

and  the  time  that  a commitment  is  required? 

211  Minutes  - e.g.,  in  deciding  to  shoot  an 
approaching  unidentified  plane. 

212  Hours  - e.g.,  in  the  Pueblo  incident. 

213  Days  - e.g.,  in  a NATO  mobilization  decision. 

214  Months  - e.g.,  in  planning  a reconnaissance 
system. 

22  Analytic  Process 

221  Number  of  input  sources  - How  many  sources 
will  provide  input  data? 

2211  Single  for  all  inputs  - All  inputs  will 
come  from  a single  source. 

2212  Single  for  each  input  - There  will  be 
several  input  sources,  but  only  one 
source  for  each  input. 

2213  Several  preference  sources  - There  will 
be  several  (e.g. , preference  value) 
input  sources  but  only  a single  source 
for  substantive  input. 

2214  Several  substantive  sources  - There  will 
be  several  sources  for  substantive 
(e.g. , uncertainty)  input  but  only  a 
single  source  for  value  input. 

« 

I . 2215  Several  for  both  - There  will  be  several 

I f substantive  and  preference  sources. 
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222  Analytic  Team 


2221  Number  of  team  members  - How  many  people 
will  actually  perform  the  analysis? 

22211  Single  - e.g.,  the  decision  maker 
himself,  a member  of  his  staff, 
or  an  outside  consultant. 

22212  Multiple  - e.g.,  a group  of  staff 
members  or  a horizontal  task  force 
comprised  of  several  people  of 
equal  rank  in  the  organization. 

2222  Type  of  analyst  - Who  will  actually 
perform  the  analysis? 

22221  The  decision  maker  himself. 

22222  A member  of  the  decision  maker's 
staff . 

22223  An  outside  consultant. 

2229  Other  distinctions. 

223  Constraints  on  analytical  method  - What 
established  procedures  constrain  the  analytic 
method? 

2230  None  - The  analysis  is  unconstrained. 

2231  Required  - The  analysis  is  required  to 
conform  to  established  procedures. 

2239  Other 

224  Documentation  - What  procedures  constrain  re- 
porting and  documentation? 

2240  None 

2241  Required  - A certain  form  of  documenta- 
tion is  required. 

2249  Other 


23  Organizational  Process  - These  dimensions  deal 
with  the  way  in  which  a particular  decision  is 
articulated  within  the  organization  and  the  way 
the  decision  is  initiated,  used,  and  controlled. 


231  Initiation 

2311  How  is  the  analysis  initiated? 

23111  Spontaneously  in  response  to 
unanticipated  developments. 

23112  Scheduled  ahead  of  time. 

23119  Other 

2312  Who  initiates  the  decision? 

23121  The  decision  maker 

23122  His  staff 

23123  His  superior 

23124  A subordinate 

23125  A peer 
23129  Other 

232  Responsibility  - Where,  in  the  organization, 
is  the  responsibility  for  the  decision? 


2321  Role  of  decision  analysis  user  - What 

role  does  the  decision  analysis  user 

serve  in  the  decision  process? 

23211  Firm  decision  - He  will  make  the 
final  decision. 

23212  Tentative  decision  - He  will  make 
a decision  that  is  subject  to 
someone  else's  revision. 

23213  Recommendation  - He  will  make  a 
recommendation . 

23214  Information  - He  will  provide  in- 
formation for  someone  else's  deci- 
sion. 

23219  Other 
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2322  Dispersion  of  decision  responsibility  - 
How  is  the  responsibility  for  the  deci- 
sion dispersed? 


23220  Not 

23221  Horizontally 

23222  Vertically 
23229  Other 

233  Coordination  - To  what  degree  must  the  deci- 
sion coordinate  with  other  higher,  lower,  or 
collateral  decisions? 

2330  Not  at  all 

2331  Moderately 

2332  Very  much 

234  Justification  - Is  the  analysis  required  as 
justification  before  or  after  a decision  is 
made? 

2340  No 

2341  Pre  only 

2342  Post  only 

2343  Both  pre  and  post 

235  Controversial  - How  controversial  is  the 
decision? 

2350  Not 

2351  Moderately 

2352  Very 

236  Performance  control  - what  type  of  account- 
ability, evaluation,  or  reward  system  is  in 
operation? 

2360  None  - The  decision  maker  is  his  own 
master. 

2361  Loose 
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2362  Tight  - The  decision  maker  operates 
under  very  strict  control. 

2369  Othe’'  - e.g.  , the  decision  maker  is  con- 
trolled by  more  than  one  superior  (as 
with  the  Chief  of  Naval  Operations). 

237  Rational-actor  model  - How  well  is  the  organ- 
izational decision  process  approximated  by  a 
rational  actor  model? 

2371  Poor  approximation  . 

2372  Good  approximation  . 

238  Risk  Attitude  - Does  the  process  require 
the  consideration  of  attitude  toward  risk? 

2380  Risk  neutral  . 

2381  Risk  averse. 

2 382  Risk  seeking  . 

2389  Other  non-linear  utility  function 

239  Other  organizational  processes 

24  Decision  Maker  Characteristics 

What  are  the  characteristics  of  the  person  making 
the  decision? 

241  Position  in  organization 

2411  Authority  level  - What  authority  does 
the  decision  maker  command? 

24111  Low  - first-line  manager  (e.g.. 
Lieutenant) . 

24112  Medium 

24113  High  - top  management  (e.g.. 
General  Officer) . 

2419  Other  - e.g.,  What  type  of  organization 
does  the  decision  maker  belong  to? 

242  Personal  characteristics 
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2421  Familiarity  with  decision  analysis  - How 
familiar  is  the  decision  maker  with 
decision-analytic  techniques? 

24211  None 

24212  Little 

24213  Much 

2429  Other,  e.g.,  management  style. 

249  Other  characteristics 
Resources  Available 

251  Computational  facilities  - How  much  computa- 
tional facility  is  available  for  analysis? 

2510  None 

2511  Some  - small  computer. 

2512  Much  - powerful  computer. 

252  Staff  - What  type  of  staff  support  is  avail- 
able to  the  decision  maker? 

2520  None 

2521  Modest  - a few  or  untrained  people. 

2522  Strong  - many  competent  people. 

253  Decision-analytic  expert  - Does  the  decision 
maker  have  access  to  decision-analytic  exper- 
tise either  in-house  or  through  consultants? 

2530  None 

2531  Low 

2532  High 

254  Decision-maker  availability  - How  available 
will  the  decision  maker  be  throughout  the 
course  of  the  analysis? 

2540  Not 

2541  Little 


2542  Much 


255  Availability  of  assessors  - How  available 
will  the  assessors  be  throughout  the  course 
of  an  analysis? 

2550  Not 

2551  Low 

2552  High 

256  Dollars  available  - How  much  money  is  available 
for  the  analysis? 

2561  Less  than  $15,000. 

2562  $15,000  to  $50,000. 

2563  Greater  than  $50,000. 

259  Other  resources 

29  Other  Decision  Processes 

291  Negotiation  - Are  there  two  or  more  actors 
with  conflicting  interests? 

2910  No  - single  actor 

2911  Two 


2912  More 


1 USER'S  OPTIONS 


11  USE  DECISION  ANALYSIS 
AT  ALL? 

111  intuitive 

112  decision  analysis 
119  other 


DOLLAR  AMOUNT  OF  ANALYSIS 


INPUT  STRUCTURE 


31  UNCERTAINTY 

311  explicit  modeling 

312  time  horizon 

313  subsequent  acts 

314  event  sequency 

ordering 

315  detail  level 

316  degree  of  grouping 


ELICITATION  TECHNIQUE 

451  for  discrete 

probabi 1 ities 

452  for  continuous 

variables 

453  for  values 

454  use  group 

elicitation? 


121  low 

122  medium 

123  high 


ROLE  OF  DECISION  ANALYSIS 

131  private/public  aid 

132  prescribed/optional 

133  contingent/current 

analysis 

134  optimization/display 

135  communication 


14  ORGANIZATION 

141  analysis  source 

142  input  source 

143  "vest-pocket"  relation 
to  decision-maker? 


321  comprehensive? 

322  decomposed? 

323  partial  list? 

324  single  index? 

325  function? 


33  SPECIAL  FORMS 

331  Markov 

332  Pareto 

333  linear  programming 


INPUT  SPECIFICATION 


41  DECISION  OPTIONS 


411  specificity  of  definition 
419  other 


51  SPECIFICATION 

511  preferred  decision 

512  single  value  for 

each  option 

513  value  distributions 
519  other 


52  DISPLAY  FORMAT 


521  graphic 

522  computer 
529  other 


53  ANALYTIC  DEVICES 

531  use  simulation 
539  other  analytic 
devices 


6 MODEL  MANAGEMENT 


151  use  a decision 

analysis  expert? 

152  use  a computer? 


MODEL  APPROACH  OPTIONS 


21  COMPLEXITY  OF  ANALYSIS 


421  scenarios 

422  specific 
429  other 


43  VALUE  CRITERIA 


61  MODEL  DYNAMICS 

611  combining 

612  pooling 

613  sequential  modeling 

614  decision  option 

scanning 

615  input  iteration 


211  simple 

212  moderately  complex 

213  very  complex 


22  COMPREHENSIVE/PARTIAL  ANALYSIS 


221  comprehensive 

222  partial 


23  APPROXIMATE  ANALYSIS? 


2 30  no 
231  yes 


431  units 

432  base 

433  evaluation  date(s) 

for  time  stream 


44  INDIRECT  ASSESSMENTS 


441  conditioned  assess- 

ment model? 

442  Bayesian  updating? 

443  decomposed  assess- 

ment model? 

444  regression? 

449  other 


62  CONTINGENT  ANALYSIS  INPUT 
SEQUENCE 

621  values 

622  priors 

623  likelihood 

624  data 


Table  B-1 : ANALYTIC  TAXONOMY  - SUMMARY 
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User's  Options  - those  options  that  should  be  consid- 
ered by  the  decision  maker  rather  than  the  analyst. 


S • 
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11  Use  decision  analysis  at  all?  - Should  any  of  the 
following  decision  analytic  techniques  be  used  or 
should  intuitive  decision-making  techniques  be 
used? 

111  Intuitive  - Use  intuitive  decision-making 
devices . 

112  Decision  analysis  - Use  decision  analysis. 

119  Other  - Use  another  decision-making  tech- 
nique. 

12  Dollar  amount  of  analysis  - What  amount  of  money 
should  be  devoted  to  the  analysis  effort? 

121  Low  - less  than  $15,000. 

122  Medium  - $15,000  to  $50,000. 

123  High  - greater  than  $50,000. 

13  Role  of  decision  analysis  - What  role  should  the 
analysis  serve  in  the  decision-making  process? 

131  Private/public  aid  - Should  the  decision 
analysis  serve  as  a personal  aid  for  the  de- 
cision maker,  or  should  the  analysis  serve  a 
more  public  function? 

1311  Private 

1312  Public 

132  Prescribed/optional  - Should  the  decision 
indicated  by  the  analysis  be  prescribed,  as 
in  an  automatic  control  system,  or  should  the 
indicated  decision  be  optional? 

1321  Prescribed 

1322  Optional 

133  Contingent/current  analysis  - Should  deci- 
sions that  are  anticipated  in  the  future  be 
analyzed  in  advance,  or  should  they  be 
analyzed  only  when  the  decision  situation  has 
materialized? 
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1331  Contingent  - analyzed  in  advance. 
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1332  Current  - analyzed  when  the  decision  is 
current . 

134  Optimization/display  only  - Should  a closed 
optimization  model  be  used  [See  chapter  18  of 
Brown,  et.al.  (1974)],  regardless  of  what 
display  options  are  used?  (Display  options 
are  presented  in  A51  below. ) 

1341  Optimization 

1342  Display  only 

135  Communication  - Should  the  analyses  serve  to 
communicate  the  reasons  for  a choice? 

1350  No 

1351  Yes 

14  Organization  - How  should  this  analysis  effort  be 
organized? 

141  Analysis  source  - Who  should  take  responsi- 
bility for  performing  the  analytic  work? 

1411  Decision  maker 

1412  Staff  member 
1419  Someone  else 

142  Input  source  - Who  should  be  responsible  for 
providing  the  model  inputs? 

1421  Decision  maker 

1422  Analyst 

1423  Expert 

143  "Vest  pocket"  relation  to  decision  maker  - 

Should  the  analyst  work  in  a close,  "vest 
pocket"  relation  to  the  decision  maker? 

1430  No 

1431  Yes 
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15  Resources  - What  special  resources  should  be 
devoted  to  the  analysis? 

151  Use  a decision-analytic  expert?  - Should  a 
decision  analysis  expert  be  used  to  provide 
assistance  in  the  analytic  effort? 

1510  No 

1511  Yes 

152  Use  a computer?  - Should  a computer  be  used 
in  the  analysis? 

1520  No 

1521  Yes 

2 Model  Approach  Options  - These  options  concern  the 

types  of  decision-analytic  modeling  techniques  to  use. 

21  Complexity  of  analysis  - On  the  whole,  how  complex 
should  the  analysis  effort  be? 

211  Simple 

212  Moderately  complex  - e.g.,  on  a par  with  a 
100-node  decision  tree. 

213  Very  complex 

22  Comprehensive/Partial  analysis  - Should  the  anal- 
ysis encompass  only  a part  of  the  complete  problem 
or  purport  to  reflect  all  relevant  considerations? 

221  Comprehensive  - reflect  all  relevant  considerations. 

222  Partial  - encompass  only  a part  of  the  complete 
problem. 

2221  Inference  only  - perform  an  analysis  to 
draw  inferences  from  the  data. 

2222  Single  value  dimension  - consider  only 
one  of  the  several  relevant  dimensions 
of  value. 

2229  Other  - perform  some  other  part  of  the 
complete  analysis. 
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23  Approximate  analysis  - Should  the  analysis  be 

performed  using  only  approximations  of  the  vari- 
ables (e.g. , values,  probabilities,  decision 
options) ? 

230  No 

231  Yes 

3 Input  Structure  - those  alternative  ways  of  structuring 
the  inputs  to  the  analysis. 

31  Uncertainty  - How  should  uncertain  events  be 
structured  for  the  analysis? 

311  Explicit  modeling  - Is  uncertainty  explicitly 
encoded? 

3110  No  certain  equivalents  only. 

3111  Simple  measures  of  uncertainty  - e.g., 
credible  intervals. 

3112  Complete  probability  distributions. 

312  Time  horizon  - Over  what  period  of  time 
should  events  be  explicitly  modeled? 

3121  - Short 

3122  - Long 

313  Subsequent  acts  - How  should  acts  subsequent 
to  the  initial  choice  (e.g. , an  act  based 
upon  the  outcome  of  a test  marketing  effort) 
be  treated  in  the  analysis? 

3130  Not  explicitly  modelled. 

3131  Modelled  in  preposterior  format. 

3132  Modelled  as  events. 

3139  Other 

314  Event  sequence  ordering  - In  what  order 
should  the  events  be  structured  in  the  anal- 
ysis? 

3141  Chronological 
3149  Non-chronological 
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315  Detail  level  - What  degree  of  detail  should 
be  included  in  structuring  the  inputs? 

3151  Low  - e.g.,  binary  grouping  of  vari- 

ables. 

3152  Medium  - e.g.,  grouping  by  10  bracket 
medians . 

3153  High 

316  Degree  of  grouping  - To  what  degree  should 
uncertain  quantities  be  grouped? 

3161  Low 

3162  High 

32  Value  - How  should  valuation  be  structured  in  the 
analysis? 

321  Comprehensive  - Should  an  attempt  be  made  to 
use  a comprehensive  valuation  measure,  which 
considers  all  items  of  concern  to  the  deci- 
sion maker? 

3210  No 

3211  Yes 

322  Decomposed  - To  what  degree  should  the 
overall  value  dimension  be  decomposed  into 
its  constituent  parts  (e.g.,  decomposing 
profit  into  units,  price,  and  profit  margin)? 

3220  None 

3221  Modest 

3222  Substantial 

323  Partial  list  - Should  only  a partial  list  of 
value  criteria  be  used  in  the  analysis? 

3230  No 

3231  Yes 

324  Single  index  - Should  all  value  dimensions  be 
aggregated  into  a single  index?  If  so,  how 
should  it  be  done? 
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3240  No 
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3241  Yes 

32411  Adjustment  - Treat  subsidiary 
value  criteria  by  making  trade- 
off adjustments  to  the  main 
criterion. 

32412  Utility  - Combine  all  value  cri- 
teria into  a single  utility 
measure. 

32419  Other  - Use  another  method  to 
arrive  at  a single  valuation 
index. 

325  Function  - If  32412,  should  multi-attributed 

value  be  expressed  in  the  form  of  a mathematical 
function?  If  so,  what  function? 

3250  No 

3251  Yes 

32511  Linear 

32519  Other  function 

33  Special  forms  - Should  any  special  analytical 

paradigms  or  techniques  be  used? 

331  Markov  - Should  a sequence  of  events  be 
treated  as  a Markov  Process? 

3310  No 

3311  Yes 

332  Pareto  - Should  the  concept  of  Pareto  opti- 
mality be  used  in  considering  decision 
options? 

3320  No 

3321  Yes 

333  Linear  programming  - Should  linear  programming 
techniques  be  used  to  optimize  the  choice? 
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3330  No 


« 


3331  Yes 

339  Other  special  forms  - Should  other  special 
forms  of  analysis  be  used? 

3390  No 

3391  Yes 

4 Input  Specification  - This  category  considers  how 

inputs  should  be  specified  once  they  have  been  struc- 
tured. 

41  Decision  options  - How  should  the  decision  option 
space  be  explored? 

411  Specificity  of  definition  - How  do  the  ana- 
lyzed decision  options  compare  in  specificity 
to  the  real  decision  options? 

4111  Real  option  - Evaluate  the  action,  e.g., 
whether  to  set  out  for  Alaska  (with  the 
exact  location  determined  later) . 

4112  Narrower  variant  of  real  option  - e.g., 
go  to  Fairbanks,  Alaska. 

4113  Broader  class  - e.g.,  go  North. 

419  Other 

42  Events  - How  should  uncertain  events  be  specified 
in  the  analysis? 

421  Scenarios  - Incorporate  events  by  specifying 
scenarios. 

422  Specific  - Use  specific  events. 

429  Other  - Use  another  method  of  specifying 
events . 

43  Value  criteria  - How  should  the  valuation  be 
specified? 

431  Units  - What  should  the  units  of  value  be? 
4311  Natural  - such  as  dollars,  hours,  etc. 
4319  Other  - such  as  utiles. 
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432  Base  - What  kind  of  reference  base  should  be 
used  for  valuation?* 

4321  Fixed  zero  - Use  an  absolute  scale. 

4322  Floating  zero  - Use  a relative  or  incre- 
mental scale  (i.e.,  a scale  that  relates 
all  values  to  those  of  a single,  possibly 
hypothetical,  alternative) . 

433  Evaluation  date(s)  for  time  stream  - What 
evaluation  date  or  dates  should  be  specified 
in  the  analysis? 

4331  Present  value  - Reduce  all  values  to  the 
present  date. 

4332  Future  value  - Project  all  values  to  a 
specified  future  date. 

4333  Time  flow  - Present  the  values  as  a flow 
over  some  specified  time  period. 

4339  Other 

44  Indirect  assessments  - What  indirect  assessment 
techniques  should  be  used  to  specify  inputs? 

441  Conditioned  assessment  - the  number  of 
conditioning  tiers  (treating  uncertainty 
explicitly  by  considering,  for  each  decision 
option,  a probability  distribution  on  the 
events  that  impact  the  valuation) . 

4410  None 

4411  Few 

4412  Several 

4413  Many 

442  Bayesian  updating  - Should  Bayes'  Theorem  be 
utilized  to  process  information  bearing  on 
some  model  inputs? 


This  concept  is  clarified  in  Brown, 


et  al . 


(1974)  pp  363-4,  96. 
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4420  No 

4421  Yes 

443  Decomposed  assessment  model  - Should  any  in- 
put be  specified  by  using  a decomposed  assess- 
ment model  that  breaks  the  inputs  into  their 
constituent  parts? 

4430  No 

4431  Yes 

444  Regression  - Should  any  of  the  inputs  be 
determined  by  using  regression  analysis? 

4440  No 

4441  Yes 

449  Other  - Should  other  indirect  assessment 
techniques  be  used? 

4490  No 

4491  Yes 

45  Elicitation  technique  - What  elicitation  techniques 
should  be  used? 

451  For  discrete  probabilities 

4511  Odds  - Elicit  relative  likelihoods. 

4512  Probabilities  - Elicit  probabilities. 

4519  Other 

452  For  continuous  variables 

4521  Fixed  probability  - e.g. , fractiles  or 
credible  intervals. 

4522  Fixed  interval 
4529  Other 

453  For  values 


4531  Reference  gamble 


4532  Direct  rating 

4539  Other 

454  Use  group  elicitation  - Should  group  elic- 
itation be  used  for  probabilities  or  values? 

4540  No 

4541  Yes 

45411  Delphi 

45412  Delbecq 
45419  Other 

5 Output 

51  Specification  - What  results  should  be  presented? 

511  Preferred  decision  - Should  the  preferred  de- 
cision be  presented? 

5110  No 

5111  Yes 

512  Single  value  for  each  option  - Should  a 
single  value  (e.g.,  an  expected  value)  be 
presented  for  each  option? 

5120  No 


5121  Yes 


513  Value  distributions  - Should  a distribution 
of  value  be  presented  for  each  option? 

5130  No 
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5131  Yes 

519  Other 

Display  Format 
be  displayed? 


- How  should  the  specified  output 
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521  Graphic  - Should  a graphic  display  of  the 
analytic  results  be  prepared? 

5210  No 

5211  Yes 

522  Computer  - Should  a computer  supply  the 
display? 

5220  No 

5221  Yes 
529  Other 

53  Analytic  Devices 

531  Use  simulation  - Should  the  analysis  of  un- 
certainty be  addressed  using  Monte  Carlo 
simulation? 

5310  No 

5311  Yes 

53111  Use  step-through?  - Should  the 
analytical  device  of  step-through 
simulation  be  used  (See  Brown 

et  al. , 1974) ? 

531110  No  - Use  regular  simulation. 
531111  Yes 

53112  Number  of  trials  - How  many 
simulation  trials  should  be  used? 

531121  Few 

531122  Many 

539  Other  analytic  devices  - What  other  analytic 
devices  should  be  used? 

5391  Grouping  - Should  grouping  devices  such 
as  bracket  medians  be  used? 
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53910  No 


53911  Yes 

5392  Pruning  - Should  pruning  devices  such  as 
dominance  be  used? 

53920  No 

53921  Yes 

6 Model  Management 

61  Model  Dynamics 

611  Combining  - Should  several  models  be  linked 
together  in  series? 

6110  No 

6111  Yes 

612  Pooling  - Should  several  simultaneous  models 
be  linked  together  in  parallel? 

6120  No 

6121  Yes 

613  Sequential  modeling  - Should  the  problem  be 
modeled  sequentially? 

6130  No 

6131  Yes 

614  Decision  option  scanning  - How  should  the 
decision  option  space  be  scanned? 

6141  Complete  enumeration  - by  examining  all 
possible  decision  options. 

6142  Sequential 
6149  Other 

615  Input  iteration  - How  much  should  inputs  be 
modified  within  the  same  model  structure 
(e.g.,  sensitivity  analysis)? 
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6151  Values 
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61511  Low 

61512  Medium 

61513  High 

6152  Probabilities 

61521  Low 

61522  Medium 

61523  High 

6153  Both  values  and  probabilities 

61531  Low 

61532  Medium 

61533  High 

62  Contingent  analysis  input  sequence  ~ When  should 
the  various  inputs  be  entered  into  a contingent 
analysis  (A1331  above)? 

621  Values  - When  should  values  be  entered? 

6211  Early  - contingently,  before  the  de- 
cision is  made. 

6212  Late  - when  the  decision  is  made. 

622  Prior  probabilities 

6221  Early 

6222  Late 

623  Likelihoods  - Likelihoods  for  indicators 

6231  Early 

6232  Late 

624  Data 

6241  Early 

6242  Late 
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1 Quality  of  Decision 


3 Other  Considerations 


11  LOGIC  OF  CHOICE 


31 


ACTIVITY  PRECEDING  CHOICE 
PROCESSES 


111 

conceptual  completeness 

112 

effective  disaggregation 

311 

qood  environment 

113 

sound  predictions 

monitorinq 

114 

good  overall  logic 

312 

good  decision 

115 

scope 

identification 

119 

other 

313 

good  option  generation 

314 

good  pre-analysis  of 

anticipated  decisions 

QUALITY  OF  INPUT 

319 

other 

121 

good  data  gathering 

32  ACTIVITY  FOLLOWING  CHOICE 

122 

good  management  of  staff/ 

PROCESSES 

expertise 

123 

posing  meaningful  questions 

124 

good  overall  input  quality 

321 

good  decision  communi- 

129 

other 

cation 

322 

good  hindsight  eval- 

uation 

Time 

and  Cost 

323 

effective  implementa- 

tion 

329 

other 

21  ELAPSED  TIME 


211  short  elapsed  modeling 

212  fast  input  assignment 

213  fast  calculation 

214  fast  interpretation 

215  short  overall  net  elapsed 

time 

219  other 


33  ORGANIZATIONAL  AND  OTHER 
NON-"CHOICE  SPECIFIC" 
IMPACTS 


22  COSTS 


331 

332 


333 


339 


improved  information 
improved  command, 
control,  and 
communication 
improved  body  of 
applied  precepts 
other 


221  inexpensive  analysis 

222  inexpensive  input  assignment 

223  inexpensive  calculation 

224  inexpensive  overall 
229  other 


Table  C-1 : PERFORMANCE  MEASURES  - SUMMARY 


K: 
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1  Qualify  of  decision  - the  quality  of  the  reasoning  or 
logic  and  the  quality  of  the  inputs. 


11  Logic  of  choice  - making  choices  that  are  in 

logical  agreement  with  available  information. 

111  Conceptual  completeness  - taking  all  im- 
portant considerations  into  account,  and 
avoiding  serious  errors  of  approximation. 

112  Effective  disaggregation  - dividing  the 
problem  into  manageable  subproblems. 

113  Sound  predictions  - making  sound  inferences 
from  the  available  data. 

114  Good  overall  logic  - a summary  measure  of 
making  choices  that  are  in  logical  agreement 
with  available  information.  This  includes 
performing  a technically  competent  analysis. 

115  Scope  - addressing  the  complete  problem  (as 
opposed  to  addressing  only  the  probability  or 
utility  part  of  the  problem) . 

119  Other  logic  considerations 

12  Quality  of  input  - obtaining  complete,  accurate, 

and  timely  data  and  judgments. 

121  Good  data  gathering  - obtaining  complete, 
accurate,  and  timely  data. 

122  Good  management  of  staff /expertise  - making 
staff  assignments  in  a manner  that  makes  best 
use  of  the  available  expertise  and  facili- 
tates accurate  and  timely  data  processing. 

123  Posing  meaningful  questions  - requesting  the 
appropriate  data  and  judgments. 

124  Good  overall  input  quality  - a summary 
measure  of  obtaining  overall  appropriate, 
complete,  accurate,  and  timely  data  and 
judgments . 

129  Other  input  quality  considerations 
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2 Time  and  cost  - the  amount  of  time  and  cost  devoted  to 
the  analysis. 

21  Elapsed  time  - the  time  from  the  instant  that  a 

decision  is  recognized  to  the  time  that  a decision 
is  made. 

211  Short  elapsed  modeling  time  - keeping  the 
time  spent  on  structuring  the  model  low. 

2111  First  pass  - the  first  time  that  the 
model  is  used. 

2112  Additional  passes  - repeated  uses  of  the 
model . 

212  Fast  input  assignment  - quickly  attaching 
values  to  input  parameters  (e.g.,  values  and 
probabilities) . 

2121  First  pass 

2122  Additional  passes 

213  Fast  calculation  - quick  performance  of  the 
calculations  needed  to  determine  a recommended 
decision. 

2131  First  pass 

2132  Additional  passes 

214  Fast  interpretation  - ability  to  quickly 
interpret  the  analysis  output. 

2141  First  pass 

2142  Additional  passes 

215  Short  overall  net  elapsed  time  - fast  per- 
formance of  items  P211-P214  above. 

2151  First  pass 

2152  Additional  passes 

2153  Total  - all  passes 


219  Other  time  considerations 
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22  Costs  - The  amount  of  executive  time  and  anguish 

and  cash  expenses  involved  in  making  the  decision. 

221  Low-cost  analysis  - performing  an  analysis 
cheaply. 

2211  First  pass 

2212  Additional  passes 

222  Low-cost  input  assignment  - making  all  required 
input  assignments  cheaply. 

2221  First  pass 

2222  Additional  passes 

223  Low-cost  calculation  - performing  the  required 
calculations  cheaply. 

2231  First  pass 

2232  Additional  passes 

224  Low-cost  overall  - a low  level  of  overall 
costs. 

2241  First  pass 

2242  Additional  passes 

2243  Total  - all  passes 
229  Other  costs. 

Other  considerations 

31  Activity  preceding  choice  processes  - activities 
that  take  place  prior  to  considering  a decision. 

311  Good  environment  monitoring  - monitoring  the 
environment  for  indications  that  a problem 
exists  or  gathering  information  to  resolve 
the  problem. 

312  Good  decision  identification  - recognizing 
when  a decision  must  be  made  and  what  deci- 
sion is  needed. 

313  Good  option  generation  - generating  the 
options  for  choice. 


314  Good  pre-analysis  of  anticipated  decisions  - 
thinking  through  future  decisions  in  advance. 
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319  Other 

32  Activity  following  choice  processes. 

321  Good  decision  communication  - communicating 
the  decision  and  any  supporting  arguments 
(e.g.,  to  facilitate  justifying  the  decision). 

322  Good  hindsight  evaluation  of  decision  - 
facilitating  an  af ter-the-f act  evaluation  of 
the  quality  of  decision. 

323  Effective  implementation  - effectively  im- 
plementing the  choice. 

329  Other. 

33  Organizational  and  other  non-"choice  specific" 

impacts  - impacts  that  are  not  concerned  with 

making  a decision  choice,  but  that  affect  more 

permanent  or  organizational  properties. 

331  Improved  information  processing  - improving 
the  recall,  correlation,  and  presentation  of 
relevant  data. 

332  Improving  the  command,  control,  and  communi- 
cation properties  of  the  organization  - e.g. , 
making  managers  behave  more  responsibly. 

333  Improving  body  of  applied  precepts  - im- 
proving doctrinal  guidelines. 

339  Other. 


This  appendix  presents  a more  thorough  examination  of 
the  situations  favoring  decision  analysis  than  is  presented 
in  Section  3.1.2.  In  particular,  this  appendix  examines  how 
each  situation  category  shown  in  Tables  2-1  and  A-l  impact 
the  decision  to  use  decision  analysis.  As  in  other  sections 
of  this  report,  references  to  situation  categories  are 
preceded  by  the  letter  "S"  and  references  to  performance 
measures  are  preceded  by  the  letter  "P."  These  references 
are  explained  in  detail  in  Appendices  A and  C,  respectively. 
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Situation 

Classification 

DECISION  SUBSTANCE  (SI) 

Basic  Situation  (Sll) 


Current/Contingent 
Choice  (Sill) 


Expected  Number  of 
Occurrences  (S112) 


Relationship  to  the  Appropriate 
Amount  of  Decision  Analysis 


Current  choice  (Sllll)  favors 
the  use  of  decision  analysis 
because  it  is  certain  that  the 
analysis  could  be  used.  Since 
decision  analysis  is  a tool  to 
aid  in  decision  making,  its 
value  as  an  aid  is  reduced  in 
contingent  situations  (S1112) , 
where  it  is  uncertain  that  the 
decision  will  ever  occur. 

A decision  that  is  expected 
to  recur  (S1123,  S1124) 
supports  the  use  of  decision 
analysis  because  the  cost 
(P22)  per  analysis  is  reduced. 

The  amount  of  advantage  gained 
by  recurrence,  of  course, 
depends  upon  both  the  expected 
frequency  of  occurrence  and 
the  similarity  of  the  recurring 
situations  because  the  similarity 
will  determine  the  amount  of 
modification  needed. 

The  expected  number  of  occurrences 
is,  in  general,  closely  related 
to  the  current/contingent 
classification,  and  the  two 
categories  should  be  considered 
simultaneously.  For  instance, 
a current  decision  that  will 
occur  only  once  should  be 
considered  the  same  as  a 
contingent  decision  that  has  a 
50%  chance  of  occurring  twice. 

Both  of  these  situations  have 
an  expected  occurrence  rate  of 
one.  (The  advantages  of  a 
recurring  situation  may  be 
somewhat  reduced  if  the  situation 
is  also  characterized  by 
hindsight  monitoring,  (S166) 
below. ) 
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Situation 

Classification 


Operating/Information 
Act  (S113) 


Options  (S12) 

Broad/Narrow  (S121) 
Clear/Fuzzy  (S122) 


Complexity  of  Decision 
Options  (S123) 

Radical/Adaptive  (S124) 


Relationship  to  the  Appropriate 
Amount  of  Decision  Analysis 

This  classification  is  unimportant 
in  itself  for  determining 
the  appropriate  amount  of 
analysis,  but  it  is  indirectly 
important  because  of  its 
impact  on  other  situational 
dimensions.  For  example,  the 
maximum  option  impact  (S144) 
is  typically  smaller  and  the 
determinability  of  uncertainty 
(S162)  is  typically  lower  for 
an  informational  act  because 
the  impact  of  the  informational 
act  is  felt  through  a subsequent 
operating  act.  This  impact 
relationship  obscures  the 
relationship  between  the 
situation  category  and  the 
analytic  taxonomy. 


Unimportant 

Clearly  specified  options 
(S1221)  support  the  use  of 
decision  analysis  because 
conceptual  completeness  is 
facilitated  (Pill). 

Unimportant 


A situation  involving  a radical 
decision  (S1241)  supports  the 
use  of  decision  analysis 
because  decision  analysis  can 
improve  the  logic  of  choice 
(Pll)  in  situations  that  are 
outside  of  the  decision  maker's 
experience. 

Radical  decisions  also  tend  to 
be  difficult  decisions.  This 
feature  is  discussed  for 
(S131 ) below. 


Situation 

Classification 


Relationship  to  the  Appropriate 
Amount  of  Decision  Analysis 


Static/Dynamic  (S125)  Static  situations  (S1251) 

favor  the  use  of  decision 
analysis.  However,  when  decision 
analysis  is  favored  for  other 
reasons  in  a dynamic  situation 
(S1252) , the  dynamic  feature 
supports  the  use  of  a large 
amount  of  analysis.  Since 
decision  analysis  is  not 
particularly  good  at  providing 
conceptual  completeness  (Pill) 
in  dynamic  situations,  a good 
analysis  requires  a large 
effort . 


Decidability  (S13) 

Difficulty  of  Choice  (S131)  Difficult  choices  (S1312) 

support  the  use  of  decision 
analysis.  A basic  argument 
for  the  use  of  decision  analysis 
is  that  it  improves  the  decision 
quality  (Pi) . In  the  case  of 
a difficult  decision,  there  is 
much  room  for  improvement 
because  informal  decision 
processes  are  not  very  good  at 
addressing  difficult  decisions. 

Unfamiliarity  (S132)  Very  unfamiliar  decision 

situations  (S1322)  support  the 
use  of  decision  analysis 
mainly  because  of  logic-of- 
choice  considerations  (Pll) . 


Key  Considerations  (S133)  If  the  key  consideration 

is  option  generation  (S1331) , 
decision  analysis  should  not 
be  used.  Decision  analysis  is 
not  good  at  generating  decision 
options . 

If  the  key  consideration  is 
information  (S1332)  , no  strong 
case  can  be  made  either  in 
favor  of  or  against  decision 
analysis . 
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Situation 

Classification 


Stakes  (S14) 


Resources  Committed 


Cost  Swing  (S142) 


Relationship  to  the  Appropriate 
Amount  of  Decision  Analysis 

If  inference  (S1333)  is  the 
key  consideration,  use  of 
decision  analysis  is  slightly 
favored  because  decision 
analysis  can  help  produce 
sound  predictions  (P113) . 

If  choice  (S1334 ) is  the  key 
consideration,  decision  analysis 
is  strongly  indicated  because 
of  its  ability  to  improve 
choice  logic  (Pll) . 

The  stakes  involved  in  the 
decision  is  the  single  most 
important  classification  for 
determining  the  appropriate 
amount  of  decision  analysis. 

Since  larger  stakes  support 
more  decision  analysis,  only 
the  minimum  stakes  needed  to 
justify  any  analysis  will  be 
discussed.  Alternative  definitions 
of  stakes  itemized  below  are 
in  decreasing  order  of  accessibility 
but  in  increasing  order  of 
relevance.  In  general,  the 
higher  the  stakes,  the  less 
important  is  cost  (P22)  and 
the  stronger  the  case  for 
decision  analysis. 

(S14 1 ) R. A.  Howard's  dictum  is  that 

1%  of  the  amount  of  resources 
committed  to  the  decision 
should  be  devoted  to  the 
decision  analysis  effort. 

The  cost  swing  is  typically 
about  one  half  of  the  resources 
committed.  Accordingly,  about 
2%  of  the  cost  swing  should  be 
devoted  to  the  decision  analysis 
effort . 


» 


Situation 
Classification  * 

Value  Swing  (S143) 


Maximum  Option  Impact 


Expected  Option 
Impact  (S145) 


Expected  Irrationality 
Cost  (S146) 


Relationship  to  the  Aporopriate 
Amount  of  Decision  Analysis 

A difference  between  the 
best  and  worst  plausible 
outcomes  of  about  $1  million 
is  necessary  to  justify  any 
decision  analysis. 

(S144)  The  maximum  option  impact  is 
a more  relevant  but  less 
accessible  measure  of  stakes. 

As  a rough  rule  of  thumb,  the 
maximum  option  impact,  that 
is,  the  dollar  equivalent 
difference  in  outcome  attributable 
to  the  choice  of  the  best 
decision  over  worst  decision 
(among  those  which  might 
reasonably  be  taken) , should 
be  on  the  order  of  $100,000 
(S1442)  in  order  to  justify 
any  decision  analysis. 

The  expected  option  impact, 

that  is,  the  maximum  difference 

in  expected  value  between 

plausible  options  under  consideration, 

is  an  even  more  relevant  and 

less  accessible  measure  of 

stakes.  An  expected  option 

impact  of  about  $50,000  (S1452) 

is  required  to  justify  the  use 

of  decision  analysis. 

The  most  relevant  and  least 
accessible  measure  of  stakes 
is  the  expected  irrationality 
cost,  the  room  for  improvement 
in  the  decision  (the  concept 
of  irrationality  cost  is 
explained  more  fully  in  Watson 
and  Brown  [1975/1]).  The 
expected  irrationality  cost 
necessary  to  support  the  use 
of  decision  analysis  is  $20,000. 
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Situation 

Classification 

Difficulty  of  Net 
Valuation  (S151) 

The  Rest  of  the  Outcome 
Valuation  Dimensions 
(S151)  to  (S156 ) 

Outcome  Uncertainty  (S16) 

Number  of  Uncertainties 
(S161 ) 

Determinability  (S162) 


High/Low  Uncertainty  (S163) 
Subsequent  Acts  (S164) 


Type  of  Evidence  (S165) 
Hindsight  Monitoring  (S166) 


Relationship  to  the  Appropriate 
Amount  of  Decision  Analysis 


Difficult  valuation  (S1512) 
supports  the  use  of  decision 
analysis  because  of  the  improved 
logic  of  choice  (Pll) , which 
decision  analysis  allows. 


More  uncertainties  support 
the  use  of  more  decision 
analysis  because  of  the  ability 
of  decision  analysis  to  improve 
the  logic  of  choice  (Pll)  in 
the  face  of  uncertainty. 

The  less  determinable  the 
distributions,  the  stronger 
the  case  is  for  using  decision 
analysis,  again  because  of  the 
improved  logic  of  choice  (Pll) 
offered  by  decision  analysis. 

A high  degree  of  uncertainty 
supports  the  use  of  decision 
analysis  because  of  the  improved 
logic  of  choice  (Pll)  that 
should  result. 

The  existence  of  important 
subsequent  acts  supports  the 
use  of  decision  analysis 
because  it  can  enhance  conceptual 
completeness  (Pll)  in  that 
situation. 


Unimportant. 

A situation  that  is  characterized 
by  a high  degree  of  hindsight 
monitoring  (S1663)  promotes 
improvements  in  informal 


Unimportant. 
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Si tuation 
Classification 


Relationship  to  the  Appropriate 
Amount  of  Decision  Analysis 


decision  making  through  the 
development  of  an  improved 
body  of  applied  precepts 
(P333).  Thus,  hindsight 
monitoring  reduces  the  need  to 
use  decision  analysis. 


DECISION  PROCESS  (S2) 
Reaction  Time  (S21) 


Analytic  Processes  (S22) 

Number  of  Input  Sources 
(S221) 


Analytic  Team  (S222) 


In  general,  a longer  reaction 
time  will  support  more  decision 
analysis  because  of  the  time 
necessary  to  perform  an  analysis 
(P21).  However,  a contingent 
choice  analysis  (A1331)  is 
supported  by  a short  reaction 
time  combined  with  a long 
anticipation  time. 


Since  conventional,  intuitive 
decision  practice  has  difficulty 
organizing  input  from  a variety 
of  'sources,  and  decision 
analysis  provides  a method  for 
organizing  a diverse  set  of 
inputs,  use  of  decision  analysis 
can  improve  the  quality  of 
input  (P12) , especially  through 
the  management  of  staff  and 
expertise  (P122) ; and  the 
logic  of  choice  (Pll) , especially 
through  effective  disaggregation 
(P112) . Thus,  a large  number 
of  input  sources  (S2213,  4,  5) 
supports  the  use  of  a large 
amount  of  decision  analysis. 

Decision  analysis  is  favored 
when  a group  of  people  are 
available  to  perform  the 
analysis  (S22212)  mainly 
because  of  its  capacity  for 
effective  disaggregation 
(P112)  and  good  management  of 
staff  and  expertise  (P122) . 
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Situation 

Classification 


Relationship  to  the  Appropriate 
Amount  of  Decision  Analvsis 


Constraints  on  Analytic  Obviously,  if  an  analytic 

Method  (S223)  method  is  prescribed  (S2231) 

and  the  prescribed  method  is 
decision  analysis,  these 
prescriptions  support  using 
decision  analysis.  On  the 
other  hand,  if  a method  of 
analysis  other  than  decision 
analysis  is  prescribed,  this 
argues  against  using  decision 
analysis.  If  no  constraints 
are  in  force  (S2230),  this 
distinction  has  no  effect  on 
the  amount  of  decision  analysis 
to  use. 

Documentation  (S224)  Required  documentation  favors 

the  use  of  decision  analysis 
because  decision  analysis 
promotes  improved  communication 
(P332)  . 


Organizational  Processes  (S23) 


Initiation  (S231) 


Unimportant . 


Responsibility  (S232) 


Coordination  (S233) 
Justification  (S234) 


Decision  analysis  is  favored 
in  situations  in  which  a 
tentative  decision  (S23212)  is 
to  be  made  or  where  a recommendation 
is  to  be  made  (S23213)  because 
decision  analysis  can  provide 
gocd  decision  communication 
(P321) , effective  dissaggregation 
(P112) , and  good  management  of 
staff  and  expertise  (P122) . 

Unimportant . 

A need  to  justify  the 
decision  (S2341,  S2342,  or 
S2343)  supports  the  use  of 
decision  analysis  because  of 
the  need  for  good  decision 
communication  (P321). 
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Situation 

Classification 


Relationship  to  the  Appropriate 
Amount  of  Decision  Analysis 


i 


t- 


Controversial  (S235) 


Performance  Control  (S236) 


Rational-Actor  Model  (S237) 


Risk  Attitude  (S238) 


Decision  Maker 

Characteristics  (S24) 

Position  in  Organization 
(S241) 


Personal  Characteristics 
(S242) 


Controversial  decisions 
(S2352)  support  the  use  of 
decision  analysis  because  of 
the  need  for  good  decision 
communication  (P321). 

Tight  performance  control 
(S2362)  supports  the  use  of 
decision  analysis  because  of  a 
need  for  improved  command, 
control,  and  communication 
(P332 ) . 

Since  decision  analysis 
assumes  a rational-actor 
model,  a situation  that  is  a 
good  approximation  of  the 
model  (S2372)  supports  the  use 
of  decision  analysis. 

Decision  analysis  provides 
the  necessary  effective 
disaggregation  (P112)  in 
situations  in  which  risk 
attitude  needs  to  be  considered 
(S2381 , S2382 , S2383,  or 
S2389)  . 


A situation  with  a high- 
level  decision  maker  (S24113) 
supports  the  use  of  decision 
analysis  mainly  because  he  is 
in  a position  that  commands 
more  resources  (S25) ; accordingly, 
time  and  cost  (P2)  are  less 
important  considerations. 

Decision  analysis  is  indicated 
when  the  decision  maker  has 
a familiarity  with  decision 
analysis  (S24212  or  S24213) 
because  in  this  situation 
decision  analysis  will  promote 
effective  decision  implementation 
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Situation 

Classification 


Resources  Available  (S25) 

Computational  Facilities 
(S251 ) 

Staff  (S252) 


Decision  Analysis  Expert 
(S253 ) 


Decision  Maker  Availability 
(S254 ) 


Assessors  Available  (S255) 
Dollars  Available  (S256) 


Relationship  to  the  Appropriate 
Amount  of  Decision  Analysis 


(P323) , good  management  of 
expertise  (P122) , and  good 
overall  logic  (P114)  (since 
familiarity  will  increase  the 
chances  of  doing  the  analysis 
correctly) . 


The  use  of  decision  analysis 
is  supported  by  better 
computational  facilities 
(S2512)  because  they  enable 
less  expensive  calculation 
(P223)  . 

The  use  of  decision  analysis 
is  supported  by  the  availability 
of  a strong  staff  (S2522) 
because  such  a staff  is  more 
apt  to  perform  an  analysis 
correctly  and  improve  the 
overall  logic  of  choice  (P114). 

The  use  of  decision  analysis 
is  supported  by  the  availability 
of  a decision-analytic  expert 
(S2532)  because  such  an  expert 
is  more  apt  to  perform  an 
analysis  correctly  and  improve 
the  overall  logic  of  choice 
(P114 ) . 

A high  availability  of 
the  decision  maker  (S2542) 
supports  the  use  of  decision 
analysis  because,  without  it, 
logic  of  choice  (Pll)  and 
quality  of  input  (P12)  are 
seriously  impaired. 

Unimportant. 

High  dollar  availability 
(S2563)  favors  decision  analysis 
because  costs  (P22)  are  then 
unimportant. 


86 


Situation 

Classification 


Relationship  to  the  Appropriate 
Amount  of  Decision  Analysis 


Other  Decision  Processes 

— fsT51 

Negotiation  (S291)  Negotiation  situations 

(S2911-2)  support  the  use  of 
decision  analysis  for  reasons 
of  improved  choice  logic 
(Pll),  especially  through 
conceptual  completeness  (Pll) 
and  effective  disaggregation 
(P112) . Decision  analysis 
provides  a methodology  for 
simultaneously  accounting  for 
elements  of  the  problem  that 
otherwise  might  be  handled 
individually.  (For  example, 
an  important  amount  of  work 
done  at  Decisions  and  Designs, 
Incorporated  relates  to  negotiation 
problems . ) 
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This  appendix  extends  the  matching  concepts  of  Section 
3.1  to  the  determination  of  the  form  of  decision  analysis  to 
use.  In  particular,  this  appendix  characterizes  each  of  the 
analytic  options  highlighted  in  Table  3-2  in  terms  of  performance 
measure  generalizations,  and  it  proposes  a tentative  matching 
of  analytic  options  onto  situations.  The  appendix  shows  how 
the  taxonomies  can  be  used  at  several  illustrative  levels  of 
analytic  choice,  for  example,  deciding  to  do  conditioned 
assessment  or  deciding  to  use  the  Delphi  technique  to  aggregate 
probability  judgments.  It  should  be  repeated  that  no  attempt 
has  been  made  at  completeness  in  codifying  the  state-of-the- 
art  of  decision  analysis.  The  matching  generalizations 
presented  here  are  purely  illustrative. 

Each  analytic  choice  considered  will  be  taken  in  turn 
and,  in  each  case,  the  performance  characteristics  distinguishing 
the  options  will  be  discussed  first,  followed  by  discussion 
of  how  various  situation  characteristics  bear  on  the  choice 
between  them. 

Throughout  this  appendix,  analytic  options  references 
are  preceded  by  the  letter  "A,"  situation  references  are 
preceded  by  the  letter  "S,"  and  performance  measure  references 
are  preceded  by  the  letter  "P."  These  items  are  described 
in  detail  in  Appendix  B,  Appendix  A,  and  Appendix  C. 
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E.l  CONTINGENT  ANALYSIS  (A1331)1 


Performance  Characterization 


Contingent  choice  analysis  requires  a large  amount  of 
first-pass  time  (P2151)^  but  provides  a fast  response  on 
additional  passes  (P2152).  In  addition,  when  reaction  time 
is  short  (S211) , a contingent  choice  analysis  improves  both 
choice  logic  (Pll)  and  input  quality  (P12) , and  improves 
command,  control,  and  communication  (P332)  by  allowing  for 
incorporation  of  a senior's  values  into  the  analysis. 


Situation 

Classification 

Current/Contingent 
Choice  (Sill) 3 


Clear/Fuzzy  Options  (S122) 


Expected  Number  of 
Occurrences  (S112) 
and 

Maximum  Option 
Impact  {SI 14) 


Relationship  to  Contingent 
Choice  Analysis 

By  definition,  contingent 
analysis  requires  a 
contingent  choice  situation 
(S1112) . 

Clear  options  (S1221)  allow 
a contingent  analysis  to 
provide  a good  pre-analysis  of 
anticipated  decision  (P314) . 
Clear  options  also  enhance 
predictability  (see  below) . 

These  two  situational 
characteristics  need  to 
be  considered  together. 

In  general,  contingent 
choice  analysis  is  indicated 
in  situations  where  its  cost 
(P22)  is  justified  by  either  a 
large  number  of  occurrences 
(S1124)  or  by  a large  maximum 
option  impact  (S1444). 


^Analytic  options  are  described  in  Appendix  B. 

2 

Performance  measures  are  described  in  Appendix  C. 
"^Situation  characteristics  are  described  in  Appendix  A. 
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As  a rule  of  thumb,  a contingent 
analysis  should  not  be  performed 
unless  the  expected  number  of 
occurrences  is  at  least  one 
(S1122)  (e.g.,  occurring  once 

with  certainty  or  occurring 
twice  with  a 50%  probability) . 

To  determine  the  number  of 
occurrences  it  is  also  necessary 
to  consider  the  similarity  of 
the  occurrences;  the  more 
similar  the  situations  are, 
the  more  the  contingent  analysis 
is  justified.  (As  a rule  of 
thumb,  three  similar  recurrences 
are  roughly  equivalent  to  a 
single  identical  recurrence.) 

It  is  critical  that  the  actual 
decision  is  predictable,  in 
that  it  involves  the  same 
considerations  that  are  modelled 
for  a good  pre-analysis  (P314). 

As  a guideline,  the  expected 
option  impact  (considering 
number  of  occurrences)  should 
exceed  $10  million  (S1444)  to 
justify  the  cost  (P22)  of  a 
contingent  analysis.  See  page 
3-3  of  Brown  et  al.  (1975) 
for  an  illustration  of  this 
guideline  in  a Navy  task  force 
commander's  decision  situation. 

Reaction  Time  (S21)  A contingent  analysis  is 

recommended  where  reaction 
time  is  short  (S211) . In  this 
situation,  the  analysis  can 
improve  the  decision  quality 


(Pi)  by  providing  a logical 
framework  that  considers  all 
available  data. 

1 t 

Number  of  Input  Sources 

Contingent  choice  analyses 

(S221) 

can  improve  command,  control. 

< 

and  communication  (P332)  in 

l> 

situations  that  require  several 

If 

preference  (e.g.,  value) 

J 

's 

sources  to  be  considered 

W V 

. ~ ♦ l 

f 

(S2213) . 
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1 

1 1 

91 

A 

E. 2 OPTIMIZATION  (A1341) 


Performance  Characterization 


An  optimization  analysis  is  characterized  by  poor 
decision  quality  (PI) , especially  with  regard  to  conceptual 
completeness  (Pill) , unless  the  analysis  is  comprehensive 
(A221) . However,  optimization  promotes  a short  interpretation 
time  (P214) . 


Situation 

Classification  Relationship  to  Optimization 

Reaction  Time  (S21)  If  reaction  time  is  short 

(S211) , then  the  time  savings 
(P214)  provided  by  optimization 
is  more  important  than  the 
loss  in  decision  quality  (PI) . 

Other  If  a comprehensive  analysis 

is  performed  (A221) , then 
optimization  is  difficult, 
leading  to  lower  logic  quality 
(P114),  and  an  expensive 
analysis  (P22)  if  the  options 
are  unclear  (S1222)  . 

A controversial  decision 
(S2352)  may  favor  display 
(A1342) , but  not  to  the  exclusion 
of  optimization  (A1341) . 
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E.3  USE  A COMPUTER  (A1521) 


A computerized  analysis  tends  to  involve  high  first- 
pass  calculation  cost  (P2231)  and  long  first-pass  calculation 
time  (P2131) , except  for  a complex  analysis  (A213).  However, 
a computer  analysis  generally  involves  a low  level  of  subsequent- 
pass  calculation  cost  (P2232)  and  short  subsequent-pass 
calculation  time  (P2132)  . 


Situation 

Classification  Relationship  to  Computer  Use 

Expected  Number  of  Since  a computer  offers 


Occurrences  (S112) 


Maximum  Option  Impact 


Reaction  Time  (S21) 


Staff  Available  (S252) 


Other 


t 

» 

t.  %. 


's 


cheaper  and  faster  performance 
on  subsequent  passes,  use  of  a 
computer  is  supported  by  a 
large  expected  number  of 
occurrences  (S1124) . 

(S144 ) A large  maximum  option 

impact  (S1444)  justifies  a 
large  cost  (P22)  and  indicates 
a complex  model  (A213) . Both 
of  these  conditions  support  a 
computerized  analysis. 

Since  a computer  analysis 
takes  a long  time  to  build, 
days  of  anticipatory  reaction 
time  (S213)  are  needed. 

However,  since  a computerized 
analysis  runs  quickly,  only 
minutes  of  execution  reaction 
time  (S211)  are  needed. 

Some  staff  support  (S2521) 

is  needed  to  program  a computerized 

analysis. 

The  costs  (P22)  and  time 
(P21 ) of  a computerized  analysis 
are  not  significantly  worse 
than  the  alternative  of  a non- 
computerized analysis  if  the 
situation  has  clear,  complex 
options  (S1221,  S1236) , many 
determinable  uncertainties 
(S1613,  S1623) , and  many 
measurable  values  (S1523, 

S1532)  . 
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A computer  is  also  recommended 
in  conjunction  with  a complex 
analysis  (A213)  and  with 
simulation  (A5311) . 
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E. 4 COMPLEXITY  OF  ANALYSIS  (A21) 


Performance  Characterization 


A complex  analysis  can  sometimes  (for  instance,  see 
below)  provide  a good  choice  logic  (Pll)  and  input  quality 
(P12).  However,  a complex  analysis  takes  a long  time  (P21) 
and  is  expensive  (P22).  In  addition,  a complex  analysis 
provides  for  good  decision  communication  (P321)  by  explicitly 
modelling  a large  number  of  important  factors. 


Situation 

Classification 

Difficulty  of  Choice  (S131) 

Maximum  Option  Impact  (S144) 

Outcome  Valuation 
Timing  (S155) 

Subsequent  Acts  (S164) 

Reaction  Time  (S21) 

Number  of  Input  Sources 
(S221) 


Relationship  to  Comolexity 
of  Analysis 

Choice  logic  (Pll)  and 
input  quality  are  important  in 
a difficult  choice  situation 
(S1312).  If  the  situation  is 
also  one  in  which  a complex 
analysis  contributes  to  choice 
logic  and  input  quality,  a 
complex  analysis  is  recommended. 

A large  maximum  option  impact 
(S1444)  justifies  a high  cost 
(P22)  to  improve  choice  logic 
(Pll).  A complex  analysis  is 
supported  if  it  contributes  to 
improved  choice  logic. 

A complex  analysis  can 
improve  choice  logic  (Pll) 
and  input  quality  (P12)  if  the 
timing  is  late  (S1552) . 

A complex  analysis  can 
improve  choice  logic  (Pll)  and 
input  quality  (P12)  if  subsequent 
acts  have  a high  impact  on  the 
decision  outcome  (S1643) . 

A complex  analysis  requires 
much  time  (S213  or  S214) . 

A complex  analysis  can 
improve  choice  logic  (Pll) 
and  input  quality  (P12)  if 
there  are  several  input  sources 
(S2213 , S2214  or  S2215) . 
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Justification  (S234) 


Controversial  (S235) 


Decision  Maker's 
Familiarity  with 
Decision  Analysis 
(S2421) 


Computational  Facilities 
Availability  (S251) 


Staff  Available  (S252) 


Decision  Maker 

Availability  (S254) 


Assessor  Availability 
(S255 ) 


Dollars  Available  (S256) 


1 


A situation  that  requires 
justification  (S2341,  S2342, 
S2343)  requires  good  decision 
communication  (P321)  and  thus 
supports  a complex  analysis. 

Controversial  decisions 
(S2352)  require  good  decision 
communication  (P321)  and  thus 
support  complex  analyses. 

A decision  maker  who  is  very 
familiar  with  decision 
analysis  (S24213)  can  improve 
choice  logic  (Pll)  and  input 
quality  (P12)  by  using  a 
complex  analysis. 

A powerful  computer  (2512) 
can  reduce  the  computational 
cost  (P223)  of  a complex 
analysis  and  thus  favors  its 
use. 

A strong  staff  (S2522)  is 
needed  to  perform  a complex 
analysis  in  order  to  minimize 
the  risk  of  an  erroneous 
analysis  and  increase  the 
overall  choice  logic  (P114) . 

A complex  analysis  can 
improve  input  quality  (P12) 
when  a decision  maker  is 
unavailable  (S2540) . The 
complex  analysis  can  perform 
the  disaggregation  that  the 
decision  maker  might  otherwise 
do  informally. 

Assessors  must  be  very 
available  (S2552)  in  order 
for  a complex  analysis  to 
provide  good  overall  input 
quality  (P124) . 

Because  of  its  cost  (P22)  , 
a complex  analysis  requires  a 
large  dollar  availability 
(S2563 ) . 
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E. 5 PARTIAL  ANALYSIS  - INFERENCE  ONLY  (A2221) 


Performance  Generalization 


An  inference  analysis  is  limited  in  scope  (P 1 1 5 ) to 
addressing  uncertainty  (not  values  or  options).  Consequently, 
an  inference  analysis  costs  less  (P22)  than  a full  analysis. 


Situation 

Classification 


Relation  to  Inference  Only 


Key  Consideration  (S133) 


Clear/Fuzzy  Options  (S122) 


An  "inference  only"  analysis 
can  provide  the  conceptual 
completeness  (Pill) , effective 
disaggregation  (P112) , and 
sound  predictions  ( PI 1 3 ) that 
are  necessary  when  the  key 
consideration  is  inference 
(S1333) . 

An  inference  model  can 
improve  conceptual  completeness 
(Pill)  with  fuzzy  options 
(S1221) . 


Maximum  Option  Impact  (S144) 


An  "inference  only"  analysis 
is  a reduced  scope  (P115) 
consistent  with  reduced  cost 
(P22) , and  thus  requires  a low 
threshold  of  maximum  option 
impact  (S1442) . 


Difficulty  of  Net 
Valuation  (S151) 


Since  an  "inference  only" 
analysis  does  not  model 
valuation,  conceptual  completeness 
(Pill)  will  be  poor  unless  the 
valuation  is  easy  (S1510). 


Uncertainty  Determinabilitv 
(S162) 


An  inference  model  can 
improve  choice  logic  (Pll) 
when  uncertainties  are  indeterminable 
(S1621) . 
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E . 6 APPROXIMATE  ANALYSIS  (A231) 


Performance  Generalization 


An  approximate  analysis  can  be  done  quickly  (P21)  and 
inexpensively  (P22) , but  it  lacks  logic  of  choice  (Pll) 
and  quality  of  input  (P12) . 


Situation 

Classification 


Difficulty  of  Choice 
( SI  3 1 ) 


Maximum  Option  Impact 
(S144 ) 


Computational  Facilities 
Available  (S251) 


Staff  Available  (S252) 


Relationship  to  Approximate 
Analysis 

An  easy  choice  (S1311)  does 
not  require  much  improvement 
in  decision  quality  (PI)  and 
thus  supports  the  use  of  an 
approximate  analysis. 

Since  an  approximate  analysis 
can  be  cheap  (P22) , only  a 
low  maximum  option  impact 
(S1442)  is  necessary  to  justify 
an  approximate  analysis. 

An  approximate  analysis 
is  not  complex  and  is 
recommended  when  no  computer 
is  available  (S2510). 

An  approximate  analysis  is 
more  difficult  than  a complete 
one,  and  thus  an  approximate 
analysis  requires  a strong 
staff  (S2522)  in  order  to 
control  the  approximation 
error  and  effect  good  overall 
choice  logic  (P114) . 
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E. 7 SHORT  TIME  HORIZON  (A3121) 


Performance  Generalization 

A short  time  horizon  switches  attention  from  modelling 
uncertainty  to  modelling  value  in  that  the  value  model 
subsumes  uncertainties  that  are  beyond  the  modelled  time 
horizon. 


Situation  Relationship  to  Short 

Classification  Time  Horizon 

Difficulty  of  Net  Difficult  valuation  (S1512) 

Valuation  (S151)  supports  the  use  of  a short 

time  horizon,  since  a short 
horizon  focuses  attention  on 
valuation . 

Valuation  Timing  (S155)  By  definition,  a short  time 

horizon  should  be  used  in  an 
early  timing  situation  (S1551) . 

Low  (S1631) , determinable 
(1623)  uncertainties  support 
the  use  of  a short  time 
horizon,  since  a short 
horizon  focuses  attention 
away  from  uncertainty. 

Low  assessor  availability 
(S2551)  and  dollar  availability 
(S2561)  will  tend  to  indicate 
a simple  analysis  (A211) , 
which  can  often  usefully 
incorporate  a short  time 
horizon. 

I 


Uncertainty  Determinability 
(S162 ) 
and 

High/Low  Uncertainty 
(P163) 


Availability  of  Assessors 
(S255 ) 
and 

Dollars  (S256) 


E . 8 ACTS  AS  EVENTS  (A3132) 


Performance  Characterization 


Modelling  acts  as  events  reduces  cost  (P22)  and  promotes 
conceptual  completeness  (Pill) , but  may  make  posing  meaningful 
questions  (P123)  difficult  compared  with  preposterior 
modelling  (A3131) . However,  omitting  an  explicit  modelling 
of  subsequent  acts  (A3130)  reduces  cost  even  more. 


Situation 

Classification 


Impact  of  Subsequent 
Acts  on  Outcome  (S164) 


Other 


Relationship  to  Acts  as  Events 

A high  impact  of  subsequent 
acts  (S1642  or  S1643)  supports 
at  least  some  modelling  of 
subsequent  acts  (but  preposterior 
even  more  than  acts-as-events) . 

Modelling  subsequent  acts 
as  events  promotes  conceptual 
completeness  (Pill)  when  it  is 
difficult  to  model  informational 
events  or  the  subsequent 
analysis  needed  for  preposterior 
modelling  [see  Brown  (1975)]. 


E. 9 DECOMPOSED  VALUATION  (A3222) 


Performance  Characterization 


Sometimes  decomposed  valuation  can  improve  choice  logic 
(Pll),  especially  through  effective  disaggregation  (P112) , 
and  input  quality  (P12) . However,  decomposed  valuation  is 
slow  (P21)  and  expensive  (P22).  Also,  decomposed  valuation 
is  limited  in  scope  (P115)  to  valuation. 


Situation 

Classification 

Maximum  Option  Impact  (S144) 


Difficulty  of  Net  Outcome 
Valuation  (S151) 


Reaction  Time  (S21) 


Number  of  Input  Sources 
(S221) 


Controversy  (S235) 


Computational  Facilities 
(S251) 


Relationship  to  Decomposed 
Valuation 


As  a rule  of  thumb,  a decomposed 
valuation  analysis  does  not 
require  high  stakes,  but  for 
larger  stakes  (e.g.,  S1444) 
costs  (P22)  are  less  important 
and  decomposed  valuation  is 
even  more  strongly  indicated. 

Decomposed  valuation  can 
help  provide  conceptual 
completeness  (Pill)  and  effective 
disaggregation  (P112)  when 
outcome  valuation  is  difficult 
(S1512 ) . 

Decomposed  valuation  requires 
hours  of  reaction  time  (S212)  , 
but  longer  reaction  time  is 
desirable. 

Decomposed  valuation  can 
help  effective  disaggregation 
(P112)  in  a situation  with 
multiple  preference  sources 
(S2213  or  S2215) . 

Decomposed  valuation  can 
aid  good  communication  (P321) 
for  controversial  decisions 
(S2352 ) . 

A computer  (S2511  or  S2512) 
is  useful,  but  not  necessary, 
for  decomposed  valuation. 
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E. 10  SINGLE,  ADJUSTED  INDEX  (A32411) 


Performance  Characterization 

A single,  adjusted  index  promotes  good  decision  communication 
(P321)  because  the  number  is  interpretable  by  a nontechnical 
user  and  the  number  has  more  information  content  than  a 
dimensionless  utility.  If  value  cannot  be  reduced  to  a 
single  index,  then  insufficient  conditions  exist  for  a 
choice  [the  situation  lacks  conceptual  completeness  (Pill)]. 

An  adjusted  index  is  limited  in  scope  (P115)  to  valuation. 


Situation 

Classification 

Decision  Justification 
(S234 ) 


Relationship  to  Single, 

Adjusted  Index 

An  adjusted  index  can  provide 
the  communication  (P321) 
needed  to  justify  a decision 
(S2341,  S2342  or  S2343)  . 


Controversy  (S235) 


An  adjusted  index  can  help 
provide  the  communication 
(P321)  needed  for  a controversial 
decision  (S2352) . 


u 

t 
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E. 11  LINEAR  VALUE  FUNCTION  (A32511) 


Performance  Characterization 


A linear  value  function  promotes  good  decision  communication 
(P321)  because  it  is  commonly  used  and  readily  understood. 
However,  a linear  value  function  lacks  conceptual  completeness 
unless  certain  utility  conditions  are  met.  Specifically, 
value  dimensions  should  be  additively  independent  and  linear. 

A linear  value  function  is  limited  in  scope  (P115)  to  valuation. 


Situation 

Classification 


Decision  Justification 
(S234 ) 


Staff  Available  (S252) 


Assessor  Availability 
(S255) 


Relationship  to  Linear 
Value  Function 


A linear  value  function 
can  help  provide  the 
communication  (P321)  needed  to 
justify  a decision  (S2341, 

S2342  or  S2343) . 

Since  a linear  value  function 
requires  no  specialized  skills, 
it  is  indicated  when  no  staff 
is  available  (S2520). 

Since  a linear  value  function 
requires  no  unusually 
difficult  assessments,  only  a 
low  level  of  assessor  availability 
(S2551)  is  needed. 
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E.  12  MARKOV  (SPECIAL  FORM)  (A3311) 


Performance  Characterization 

A Markov  analysis  is  a compact  analysis,  which  enables 
inexpensive  input  assignment  (P222)  and  inexpensive  calculation 
(P223) . However,  since  a Markov  analysis  is  restricted,  it 
may  impair  conceptual  completeness  (Pill) , and,  because  it 
requires  special  skill,  a Markov  analysis  is  an  expensive 
analysis  (P221) . A Markov  analysis  is  also  restricted  in 
scope  (P115)  to  uncertainty. 


Situation 

Classification 

Key  Consideration  (S133) 

Difficulty  of  Net  Outcome 
Valuation  (S151) 

Other 


Relationship  to  Markov 


Analysis 


Markov  analysis  only  addresses 
probabilities;  therefore,  it 
is  indicated  when  inference  is 
the  key  consideration  (S1333) . 


Since  a Markov  analysis 
does  not  contribute  to 
valuation,  it  is  recommended 
when  valuation  is  easy  (S1510) . 


A Markov  analysis  requires 
a special  environmental  structure. 
Specifically,  it  requires  that 
a time  series  of  stochastically 
changing  discrete  variables, 
and  their  associated  transition 
availabilities,  follow  some 
systematic  pattern. 


E. 13  PARETO  (SPECIAL  FORM)  (A3321) 


Performance  Characterization 


A Pareto  analysis  can  improve  conceptual  completeness 
(Pill)  by  allowing  a simultaneous  analysis  of  the  joint 
utility  function  of  the  parties  involved.  On  the  other 
hand,  a Pareto  analysis  may  reduce  conceptual  completeness 
(Pill)  because  it  does  not  accommodate  the  dynamic  social 
aspects  of  negotiation. 

Situation 

Classification  Relationship  to  Pareto  Analysis 

Negotiation  (S291)  Pareto  analysis  addresses 

the  negotiation  problem  and 
thus  requires  a negotiation 
situation  (S2911  or  S2912) . 


E. 14  LINEAR  PROGRAMMING  (SPECIAL  FORM)  (A3331) 


Performance  Characterization 


When  a computer  is  available  (S2511  or  S2512),  linear 
programming  can  provide  a low-cost  overall  analysis  (P224) 
and  low-cost  calculation  (P223) , especially  in  subsequent 
passes  (P2242  and  P2232),  and  can  enable  fast  calculation 
(P213)  and  short  elapsed  modelling  time  (P211) . However, 
linear  programming  is  slow  (P21)  and  expensive  (P22)  without 
a computer  (S2510).  In  addition,  linear  programming  will 
reduce  conceptual  completeness  (Pill)  unless  restrictive 
conditions  are  met.  Specifically,  the  objective  function 
that  is  to  be  maximized  or  minimized  and  the  constraints 
must  be  linear.  Also,  linear  programming  can  enable  low-cost 
input  assignment  (P222),  because  it  requires  little  involvement 
of  the  decision  maker,  but  it  is  poor  at  posing  meaningful 
questions  (P123)  and  does  not  permit  disagqregation  (P112) 
of  value  or  uncertainties. 


Situation 

Classification 

Clear/Fuzzy  Options  (S122) 
Option  Complexity  (S123) 

Difficulty  of  Net 

Outcome  Valuation  (S151) 


Amount  of  Uncertainty  (S163) 


Relationship  to  Linear 
Programming 

Linear  programming  requires 
clear  options  (S1221) . 

Linear  programming  addresses 
vector  options  (S1235  and 
S1236)  and  is  most  useful  in 
large  vector  (S1236)  situations. 

Linear  programming  does 
not  contribute  anything 
to  determining  valuation  and 
does  not  permit  disaggregation 
(P112) . Thus,  the  situation 
must  be  characterized  by  easy 
outcome  valuation  (S1510). 

Linear  programming  assumes 
certainty  and  thus  requires  a 
low  amount  of  uncertainty 
(S1630  or  S1631) . 


r 


Computational  Facilities 
Available  (S251) 


Staff  Available  (S252) 


Decision  Maker  Availability 
(S254 ) 


Linear  programming  requires 
a computer  (S2512)  because 
it  is  slow  (P21 ) and  expensive 
(P22)  without  one. 

Linear  programming  requires 
a modest  staff  (S2521) , but  a 
strong  staff  is  better  (S2522) , 
to  ensure  an  accurate  analysis 
and  good  overall  logic  (P114) . 

Linear  programming  does 
not  require  much  decision 
maker  availability  (S2540)  for 
interaction.  This  keeps  the 
input  assignment  costs  low 
(P222)  . 


Other 


Linear  programming  assumes 
a structured,  linear  environment. 
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E. 15  FLOATING  ZERO  BASE  (A4322) 


Performance  Characterization 

A floating  zero  base  provides  a compact  analysis.  This 
allows  for  low-cost  input  assignment  (P222)  (because  fewer 
inputs  are  required),  low-cost  calculation  (P223)  and  an 
inexpensive  overall  analysis  (P224).  However,  a floating 
zero  analysis  is  less  conceptually  complete  (Pill)  if  utility 
is  non-linear,  and  is  difficult  to  communicate  (P321) . In 
addition,  a floating  zero  analysis  requires  more  assessment 
skill,  which  may  produce  an  expensive  analysis  (P221).  Also 
a floating  zero  base  is  limited  in  scope  (P115)  to  valuation. 


Relationship  to  Floating 
Zero  Base 


Because  floating  zero  is 
inexpensive  (P22) , it  requires 
only  a modest  maximum  option 
impact  (S1442) . 

Floating  zero  requires 
easy  valuation  (S1510) 
because  it  is  difficult,  and 
hence  expensive  (P22) , to 
apply  unless  valuation  is 
easy. 

Floating  zero  is  recommended 
when  decision  justification 
is  not  needed  (S2340)  because 
floating  zero  is  not  good  at 
providing  the  communication 
(P321)  needed  for  justification. 

Floating  zero  is  difficult 
and  thus  it  requires  a strong 
staff  (S2522 ) to  ensure  good 
overall  logic  (P114). 


f 
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Situation 

Classification 

Maximum  Option  Input  (S144) 


Difficulty  of  Net  Outcome 
Valuation  (S151) 


Decision  Justification 
(S234 ) 


Staff  Available  (S252) 


E . 16  CONDITIONED  ASSESSMENT  (A441) 


Performance  Characterization 

Conditioned  assessment  sometimes  promotes  logical 
choice  (Pll),  especially  through  effective  disaggregation 
(P112) , and  quality  input  (P12)  (see  below) . However, 
conditioned  assessment  is  time  consuming  (P21)  and  costly 
(P22)  . 


Situation 

Classification 


Relationship  to  Conditioned 
Assessment 


Maximum  Option  Impact  (S144) 


Determinability  of  Outcome 
Uncertainty  (S162) 


The  stakes  threshold  for 
performing  conditioned  assessment 
is  low  (S1442)  but,  because  of 
cost  of  conditioned  assessment 
(P22) , the  larger  the  stakes 
(S144),  the  stronger  the  case 
for  using  conditioned  assessment. 

Conditioned  assessment 
promotes  logical  choice 
(Pll) , especially  through 
effective  disaggregation 
(P112) , and  input  quality 
(P12)  when  outcome  uncertainty 
is  indeterminable  (S1621) . 


Reaction  Time  (S21) 


At  least  hours  of  reaction 
time  (S212)  are  needed  for 
conditioned  assessment  but  a 
longer  reaction  time  is  desirable 
(S213  or  S214)  because  elapsed 
analysis  time  (P21)  is  of  less 
concern  with  a long  reaction 
time . 


Number  of  Input  Sources 
(S221) 


Conditioned  assessment  can 
improve  input  quality  (P12) , 
especially  through  good  management 
of  staf f/expertise  (P122)  when 
there  are  several  substantive 
sources  (S2214  or  S2215)  . 


r.  *. 
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Computational  Facility 
Available  (S251) 


A computer  (S2511,  S2512) 
is  useful  for  conditioned 
assessment. 


109 


Staff  Available  (S252) 


A modest  staff  (S2521)  is 
necessary  but  a strong  staff 
is  preferred  (S2522)  in  order 
to  ensure  good  overall  logic 
(P114) . 
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E . 17  BAYESIAN  PROBABILITY  UPDATING  (A4421) 


Performance  Characterization 


A Bayesian  probability  updating  model  promotes  effective 
disaggregation  (P112)  and  a low-cost  overall  analysis  (P224). 
However,  a Bayesian  updating  model  may  lead  to  poor  input 
quality  (P124)  because  it  is  often  difficult  to  elicit  the 
necessary  prior  probabilities  and  likelihood  functions. 

Also,  Bayesian  updating  is  limited  in  scope  (P115)  to  inference. 


Situation 

Classification 

Key  Consideration  (S133) 

Staff  Available  (S252) 
Other 


Relationship  to  Bayesian 
Probability  Updating 

Bayesian  probability  updating 
is  recommended  when  the  key 
consideration  is  inference 
(S1333)  because  Bayesian 
updating  addresses  inference. 

Bayesian  updating  is  difficult. 
Therefore,  a strong  staff 
(S2522)  is  needed. 

Bayesian  updating  assumes 
a perceptual  structure  that 
includes  prior  probabilities 
and  likelihood  functions. 
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E . 18  REFERENCE  GAMBLE  (A4531) 
(ELICITATION  TECHNIQUE  FOR  VALUES) 


♦ 


Performance  Characterization 

The  reference  gamble  elicitation  technique  can  provide 
conceptual  completeness  (Pill) , but  the  technique  makes  it 
difficult  to  pose  meaningful  questions  (P123).  In  addition, 
the  reference  gamble  technique  rates  low  on  decision  communication 
(P321) , data  gathering  ( P 1 2 1 ) , and  involves  high  input 
assignment  cost  (P222) . Also,  the  reference  gamble  technique 
is  limited  in  scope  (P115)  to  valuation. 


Situation 

Classification 

Decision  Justification 
(S234 ) 


Staff  Available  (S252) 


Assessor  Availability 
(S255) 


Relationship  to  Reference 
Gamble  Technique 

Although  the  reference 
gamble  technique  scores 
low  on  decision  communication 
(P321) , an  important  performance 
measure  for  decision  justification, 
its  demonstrably  superior 
conceptual  completeness  (Pill) 
makes  the  reference  gamble 
technique  good  for  decision 
situations  requiring  justification 
(S2341 , S2342  and  S2343) . 

Reference  gamble  elicitation 
is  difficult.  Therefore,  a 
strong  staff  (S2522)  is  required 
to  ensure  good  overall  logic 
( P 1 1 4 ) . 

The  reference  gamble  technique 
requires  a high  degree  of 
assessor  availability  (S2552)  . 
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E. 19  DELPHI  (A45411) 

(GROUP  ELICITATION  TECHNIQUE  FOR  PROBABILITIES) 


Performance  Characterization 


The  Delphi  technique  involves  pooling  the  opinions  of 
several  probability  assessors  with  limited  interaction. 


Situation  Relationship  to 

Classif ication  Delphi  Technique 

Number  of  Input  Sources  The  Delphi  technique  addresses 

(S221)  the  problem  of  several 

uncertainty  sources  (S2214). 

Other  The  Delphi  technique  limits 

interaction  among  assessors 
and  provides  the  assessors 
with  anonymity.  Thus,  the 
Delphi  technique  removes 
inhibition  and  allows  better 
use  of  staff  expertise  (P122) 
where  the  status  of  the  assessors 
is  heterogeneous.  However,  if 
data  is  heterogeneous,  then 
the  Delphi  technique  reduces 
input  quality  (P12)  by  inhibiting 
useful  interactions.  See 
Fischer  (1975)  for  further 
information  on  the  Delphi 
technique . 
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E.  20 


SIMULATION  (ANALYTIC  DEVICE)  (A5311) 


Performance  Characterization 


Simulation  is  expensive  when  compared  with  doing  nothing, 
but  it  provides  less  costly  analysis  (P221)  and  less  costly 
calculation  (P223)  than  an  identically  structured  extensive 
form  analysis.  On  the  other  hand,  simulation  communicates 
less  effectively  (P321)  than  extensive  form.  In  addition, 
simulation  lacks  conceptual  completeness  (Pill)  due  to 
errors  of  approximation,  and  simulation  is  restricted  in 
scope  (P115)  to  probabilities. 


Situation 

Classification  Relationship  to  Simulation 

Maximum  Option  Impact  (S144)  Since  simulation  is  expensive, 

it  is  indicated  in  situations 
with  large  stakes  (S1444). 

Other  Simulation  may  be  indicated 

in  conjunction  with  a complex 
structure  (A213)  or  a conditioned 
assessment  model  (A441) . 
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E . 21  STEP-THROUGH  SIMULATION  (A531111) 
(ANALYTIC  DEVICE) 


Performance  Characterization 

Compared  with  regular  simulation  (A531110) , step- 
through  simulation  provides: 

o greater  conceptual  completeness  (Pill), 

o lower-cost  overall  modelling  (P224)  and 

input  assignment  (P222)  but  greater  demands 
on  the  decision  maker  (for  the  same  level  of 
conceptual  completeness) , and 

o better  decision  communication  (P321)  (because 
step-through  requires  the  decision  maker  to 
get  involved  in  the  analysis) . 


Situation 

Classification 


Outcome  Valuation  Timing 
(S155) 


Determinability  of 
Uncertainty  (S162) 


Decision  Justification 
(S234 ) 


Staff  Available  (S252) 


Relationship  to  Step- 
Through  Simulation 

Step-through  simulation  is 
adapted  to  improving  conceptual 
completeness  (Pill)  for  chronological 
sequences,  and  chronological 
sequences  are  significant  only 
in  situations  with  late  evaluation 
dates  (S1552) . 

Step-through  simulation 
improves  conceptual  completeness 
(Pill)  most  when  uncertainties 
are  indeterminable  (S1621) . 

Because  step-through  simulation 
involves  the  decision  maker  in 
the  analysis,  it  improves  the 
communication  (P321)  needed 
for  justification  (S2341, 

S2342,  S2343) . 

Step-through  simulation  is 
a difficult  technique,  and  it 
requires  a strong  staff  (S2522) 
to  ensure  good  overall  logic 
(P114)  . 
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Step-through  simulation  can 
improve  conceptual  completeness 
(Pill)  only  if  the  assessor  is 
available  (S2552) . (Step- 
through  shifts  the  weight  of 
effort  from  the  analyst  to  the 
assessor) . 


APPENDIX  F 


CONCEPTUAL  FRAMEWORK  FOR  MATCHING 


This  appendix  presents  a conceptual  framework  for  the 
matching  task.  Specifically,  an  analogy  is  drawn  between 
the  matching  task  and  the  task  of  formulating  a regression 
problem.  This  analogy  is  presented  only  as  a conceptualization 
of  the  matching  problem  and  does  not  imply  that  the  matching 
task  can  actually  be  formulated  in  a mathematical  representation. 

Formally,  we  can  visualize  the  matching  task  as  formulating 
a regression  or  other  estimating  function  of  the  form: 

A = f ( P ) = } ( S ) 


where 


A is  the  appropriate  values  (choices)  of  the  analytic 
option  vector, 

P is  the  vector  of  all  relevant  performance  require- 
ments for  a given  situation, 

S is  the  (much  larger)  vector  of  all  relevant  situation 
characteristics,  and 

f and  | are  the  appropriate  mapping  functions. 

In  general,  P and  £[  are  unavailable  because  the  size  of 
the  vector  is  too  great  for  practicality  because  the  elements 
in  the  vector  are  not  accurately  measured.  Typically,  what  we 
have  available  are  P'  and  S' , more  manageable  sets  of  estimated 
performance  requirements  and  situation  characteristics,  such 
as  those  proposed  in  our  performance  measure  and  situation 
taxonomies. 

As  in  regular  regression  practice,  we  are  faced  with  the 
problem  of  implicitly  or  explicitly  formulating  an  estimate 
of  A,  A,  of  the  form: 

A = 0 ( P'  , S'  ) 

where  0 is  an  estimate  of  the  appropriate  gapping  function 
that  minimizes  the  expected  error,  E(  A - A),  subject  to  the 
cost  of  measuring  and  implementing  0,  P' . S'. 

The  practical  counterpart  of  this  approach  is  that  our 
matching  generalizations  may  incorporate  both  performance 
measures  and  situation  characteristics. 
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